Self-indentifying one-way authentication method using optical signals

ABSTRACT

In one aspect, the present disclosure relates to a self-identifying optical transmitter for broadcasting a one-way authentication code using light-based communication. The transmitter may include a memory for storing an identifier of the transmitter, a processor for generating a data signal including an identifier of the transmitter, a modulator for receiving the data signal and generating an electrical signal, the modular generating the electrical signal by modulating the data signal. The transmitter may also include a light source for receiving the electrical signal, converting the electrical signal into an optical signal, and continuously broadcasting the optical signal as an optical data transmission stream. The optical data transmission stream may be used to verify that a receiving mobile device is near the transmitter. The transmitter may also include an optical surface for dispersing the optical data transmission stream as the optical data transmission stream is emitted from the transmitter.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims the benefit under 35U.S.C. § 120 to U.S. application Ser. No. 14/058,678 filed Oct. 21,2013, entitled “SELF-IDENTIFYING ONE-WAY AUTHENTICATION METHOD USINGOPTICAL SIGNALS,” the disclosure of which also is entirely incorporatedherein by reference.

U.S. application Ser. No. 14/058,678 claims priority under 35 U.S.C. §119(e) to U.S. Provisional Application No. 61/715,950, entitled“Self-Identifying One-Way Authentication Method Using Optical Signals”,filed Oct. 19, 2012, the contents of which are incorporated by referenceherein.

U.S. application Ser. No. 14/058,678 is a continuation-in-part of andclaims benefit under 35 U.S.C. § 120 to U.S. Utility application Ser.No. 14/019,376, entitled “Configuration and Management Of LightPositioning System Using Digital Pulse Recognition”, filed Sep. 5, 2013,now U.S. Pat. No. 9,288,293, the contents of which are incorporatedherein by reference.

U.S. Utility application Ser. No. 14/019,376, entitled “Configurationand Management Of Light Positioning System Using Digital PulseRecognition”, claims priority under 35 U.S.C. § 119(e) to U.S.Provisional Application No. 61/697,098, entitled “Configuration andManagement of Light Positioning System Using Digital Pulse Recognition”,filed Sep. 5, 2012, the contents of which are incorporated by referenceherein.

U.S. Utility application Ser. No. 14/019,376, entitled “Configurationand Management Of Light Positioning System Using Digital PulseRecognition”, is a continuation-in-part of and claims benefit under 35U.S.C. § 120 to U.S. Utility application Ser. No. 13/718,233, entitled“Method and System for Configuring an Imaging Device for the Receptionof Digital Pulse Recognition Information”, filed Dec. 18, 2012, theentire contents of which are incorporated herein by reference.

U.S. Utility application Ser. No. 13/718,233, entitled “Method andSystem for Configuring an Imaging Device for the Reception of DigitalPulse Recognition Information”, is a continuation of and claims benefitunder 35 U.S.C. § 120 to U.S. Utility application Ser. No. 13/526,656,entitled “Method and System for Configuring an Imaging Device for theReception of Digital Pulse Recognition Information,” filed Jun. 19,2012, now U.S. Pat. No. 8,334,898, the entire contents of which areincorporated herein by reference.

U.S. Utility application Ser. No. 13/526,656, entitled “Method andSystem for Configuring an Imaging Device for the Reception of DigitalPulse Recognition Information”, filed Jun. 19, 2012, now U.S. Pat. No.8,334,898, claims priority under 35 U.S.C. § 119(e) to U.S. ProvisionalApplication No. 61/639,428, filed Apr. 27, 2012 and entitled Method ForMeasuring Modulation Frequency Of A Light Source,” the entire contentsof which are incorporated herein by reference.

U.S. Utility application Ser. No. 13/526,656, entitled “Method andSystem for Configuring an Imaging Device for the Reception of DigitalPulse Recognition Information”, filed Jun. 19, 2012, now U.S. Pat. No.8,334,898, claims priority under 35 U.S.C. § 119(e) to U.S. ProvisionalApplication No. 61/635,413, filed Apr. 19, 2012 and entitled “DigitalPulse Recognition Demodulation Techniques For Light Based Positioning,”the entire contents of which are incorporated herein by reference.

U.S. Utility application Ser. No. 13/526,656, entitled “Method andSystem for Configuring an Imaging Device for the Reception of DigitalPulse Recognition Information”, filed Jun. 19, 2012, now U.S. Pat. No.8,334,898, claims priority under 35 U.S.C. § 119(e) to U.S. ProvisionalApplication No. 61/567,484, filed Dec. 6, 2011 and entitled “Systems AndMethods For Light Based Location,” the entire contents of which areincorporated herein by reference.

U.S. Utility application Ser. No. 13/526,656, entitled “Method andSystem for Configuring an imaging Device for the Reception of DigitalPulse Recognition Information”, filed Jun. 19, 2012, now U.S. Pat. No.8,334,898, claims priority under 35 U.S.C. § 119(e) to U.S. ProvisionalApplication No. 61/511,589, filed Jul. 26, 2011 and entitled “SystemUsing Optical Energy For Wireless Data Transfer,” the entire contents ofwhich are incorporated herein by reference.

U.S. Utility application Ser. No. 13/526,656, entitled “Method andSystem for Configuring an Imaging Device for the Reception of DigitalPulse Recognition Information”, filed Jun. 19, 2012 is acontinuation-in-part of and claims benefit under 35 U.S.C. § 120 to U.S.Utility application Ser. No. 13/446,520, entitled “Method And System ForTracking And Analyzing Data Obtained Using A Light Based PositioningSystem,” filed Apr. 13, 2012, now U.S. Pat. No. 8,947,513, which is acontinuation of and claims benefit under 35 U.S.C. § 120 to U.S. Utilityapplication Ser. No. 13/445,019, entitled “Single Wavelength LightSource for Use in Light Based Positioning System,” filed Apr. 12, 2012;U.S. Utility application Ser. No. 13/435,448, entitled “A Method andSystem for Calibrating a Light Based Positioning System,” filed Mar. 30,2012; U.S. Utility application Ser. No. 13/422,591, entitled “SelfIdentifying Modulated Light Source,” filed Mar. 16, 2012, now U.S. Pat.No. 8,866,391; U.S. Utility application Ser. No. 13/422,580, entitled“Light Positioning System Using Digital Pulse Recognition,” filed Mar.16, 2012, now U.S. Pat. No. 8,248,467; U.S. Utility application Ser. No.13/369,147, entitled “Content Delivery Based On A Light PositioningSystem,” filed Feb. 8, 2012, now U.S. Pat. No. 8,964,016; and U.S.Utility application Ser. No. 13/369,144, entitled “Independent BeaconBased Light Positioning System,” filed Feb. 8, 2012, now U.S. Pat. No.9,287,976.

U.S. Utility application Ser. No. 13/526,656, entitled “Method andSystem for Configuring an Imaging Device for the Reception of DigitalPulse Recognition Information”, filed Jun. 19, 2012, now U.S. Pat. No.8,334,898, is also a continuation-in-part of and claims benefit under 35U.S.C. § 120 to U.S. Utility application Ser. No. 13/446,506, entitled“Method And System For Determining the Position Of A Device In A LightBased Positioning System Using Locally Stored Maps,” filed Apr. 13,2012, now U.S. Pat. No. 8,994,799, which is a continuation of and claimsbenefit under 35 U.S.C. § 120 to U.S. Utility application Ser. No.13/445,019, entitled “Single Wavelength Light Source for Use in LightBased Positioning System,” filed Apr. 12, 2012; U.S. Utility applicationSer. No. 13/435,448, entitled “A Method and System for Calibrating aLight Based Positioning System,” filed Mar. 30, 2012; U.S. Utilityapplication Ser. No. 13/422,591, entitled “Self Identifying ModulatedLight Source,” filed Mar. 16, 2012, now U.S. Pat. No. 8,866,391; U.S.Utility application Ser. No. 13/422,580, entitled “Light PositioningSystem Using Digital Pulse Recognition,” filed Mar. 16, 2012, now U.S.Pat. No. 8,248,467; U.S. Utility application Ser. No. 13/369,147,entitled “Content Delivery Based on a Light Positioning System,” filedFeb. 8, 2012, now U.S. Pat. No. 8,964,016; and U.S. Utility applicationSer. No. 13/369,144, entitled “Independent Beacon Based LightPositioning System,” filed Feb. 8, 2012, now U.S. Pat. No. 9,287,976.

U.S. Utility application Ser. No. 13/526,656, entitled “Method andSystem for Configuring an Imaging Device for the Reception of DigitalPulse Recognition Information”, filed Jun. 19, 2012, now U.S. Pat. No.8,334,898, is also related to the following applications, filedconcurrently with U.S. Utility application Ser. No. 13/526,656, theentire contents of which are incorporated herein by reference: U.S.patent application Ser. No. 13/526,808, filed on Jun. 19, 2012, entitled“Method And System For Modifying A Beacon Light Source For Use In ALight Based Positioning System;” U.S. patent application Ser. No.13/526,788, filed on Jun. 19, 2012, entitled “Method And System ForModulating A Light Source In A Light Based Positioning System Using A DCBias;” U.S. patent application Ser. No. 13/526,812, filed on Jun. 19,2012, entitled “Device For Dimming A Beacon Light Source Used In A LightBased Positioning System;” U.S. patent application Ser. No. 13/526,781,filed on Jun. 19, 2012, entitled “Method And System For Modulating ABeacon Light Source In A Light Based Positioning System;” U.S. patentapplication Ser. No. 13/526,773, filed on Jun. 19, 2012, entitled“Method And System For Digital Pulse Recognition Demodulation;” U.S.patent application Ser. No. 13/526,814, filed on Jun. 19, 2012, entitled“Method And System For Video Processing To Determine Digital PulseRecognition Tones;” and U.S. patent application Ser. No. 13/526,779,filed on Jun. 19, 2012, entitled “Method And System For Demodulating ADigital Pulse Recognition Signal In A Light Based Positioning SystemUsing A Fourier Transform.”

The above referenced applications are hereby incorporated by referencein their entirety.

FIELD OF THE DISCLOSURE

This disclosure relates generally to a system and method fortransmitting self-identifying modulated light signals.

BACKGROUND

Indoor positioning services refers to methods where networks of devicesand algorithms are used to locate mobile devices within buildings.Indoor positioning is regarded as a key component of location-awaremobile computing and is an important element in provided augmentedreality (AR) services. Location aware computing refers to applicationsthat utilize a user's location to provide content relevant to location.Additionally, AR is a technology that overlays a virtual space onto areal (physical) space. To successfully enable AR and location awarecomputing, accurate indoor positioning is an important requirement.

Global Positioning Systems (GPS) loses significant power when passingthrough construction materials, and suffers from multi-path propagationeffects that make it unsuitable for indoor environments. Techniquesbased on received signal strength indication (RSSI) from WiFi andBluetooth wireless access points have also been explored. However,complex indoor environments cause radio waves propagate in dynamic andunpredictable ways, limiting the accuracy of positioning systems basedon RSSI. Ultrasonic techniques (US), which transmit acoustic waves tomicrophones, are another method which can be used to approximate indoorposition. They operate at lower frequencies than systems based on WiFiand attenuate significantly when passing through walls. This potentiallymakes US techniques more accurate than WiFi or Bluetooth techniques.

Optical indoor positioning techniques use optical signals, eithervisible or infrared, and can be used to accurately locate mobile devicesindoors. These are more accurate than the approaches mentionedpreviously, since optical signals are highly directional and cannotpenetrate solid objects. However this directionality limits thepotential reliability of optical signals, since difficulty in aligningthe receiver and transmitter can occur.

A variety of systems transmit a one-way authentication signal from aninfrastructure endpoint to a mobile device. Applications that make useof one-way authentication include mobile loyalty solutions, ticketing,secure access control, payments, media sharing, and social media. Acommon technology used for one-way authentication is Quick ResponseCodes (“QR codes”). QR codes are two-dimensional barcodes. Mobiledevices that include cameras can take a picture of the QR code and thendecode the information associated with the QR code.

However, QR codes have the disadvantage of being easily copied. Anyonehaving a device with a camera can take a picture of a QR code, print itout, and replicate it. For example, when QR codes are used in mobileloyalty solutions, it is possible for users to cheat the system bytaking pictures of the QR codes and submitting fake scans. Analternative emerging technology is Near-Field-Communication (NFC), whichuses radio frequency transmission to transmit an authentication messageover a short distance. However, NFC is still in the nascent stages ofadoption. Many mobile devices do not have NFC capabilities, and NFCterminals have not yet been widely deployed.

SUMMARY

Given the security issues with QR codes, and the lack of NFC-enableddevices, demand exists for a secure, one-way authentication method thatworks on mobile devices available today. This disclosure describes adevice that uses an optical signal for transmitting a secure, one-wayauthentication code. The signal is received using an optical sensorpresent on many mobile devices. This optical sensor can be an imagesensor that is part of a mobile device's camera, a photodiode, areverse-biased LED, or any other optical receiver that would be readilyunderstood by a person of ordinary skill in the art. The optical signalmay be transmitted via visible light, as most camera-enabled mobiledevices are capable of receiving information transmitted via a visiblelight signal. In some embodiments, the signal may be transmitted in theinfrared spectrum, using an IR LED or other similar emitter. For mobiledevice cameras that include infrared filters that may prevent such asignal from being received, a visible light signal is preferred.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a representation of a mobile device receiving light from a LEDlight source, according to some embodiments of the present disclosure.

FIG. 2 is a representation of a mobile device receiving multiple sourcesof light simultaneously from multiple LED light sources, according tosome embodiments of the present disclosure.

FIG. 3 is a representation of the internal components commonly found ina LED light source that is capable of being modulated to send digitaldata.

FIG. 4 illustrates information which can be optically transmitted froman LED light source.

FIG. 5 is a representation of the components which are commonly found inmobile devices which enable them to receive optical signals from LEDsources.

FIG. 6 is a representation of multiple LED light sources sending uniqueinformation to multiple mobile devices.

FIG. 7 illustrates the process of a mobile device sending identificationinformation and receiving location information via a network to aserver.

FIG. 8 illustrates the high level contents of the server which includesdatabases and web services for individual areas enabled with lightpositioning systems.

FIG. 9 illustrates the components inside the databases.

FIG. 10 illustrates the information contained in the Light IDs database.

FIG. 11 illustrates the information contained in the Maps database.

FIG. 12 illustrates the information contained in the Content database.

FIG. 13 illustrates the information contained in the Analytics database.

FIG. 14 illustrates the process of a mobile device receiving locationand content information via a light-based positioning system.

FIG. 15 is a process illustrating the background services and how theyactivate various sensors contained inside the mobile device.

FIG. 16 illustrates the process of combining multiple informationsources with a light-based positioning service.

FIG. 17 illustrates how a client accesses multiple light positioningenabled locations with multiple mobile devices.

FIGS. 18A-C are representations of a light source undergoingpulse-width-modulation at varying duty cycles, according to someembodiments of the present disclosure.

FIGS. 19A-C are representations of a light source undergoingpulse-width-modulation at varying duty cycles with a DC offset,according to some embodiments of the present disclosure.

FIG. 20 is a block diagram of a DPR modulator with a dimming controlsystem for a light source, according to some embodiments of the presentdisclosure.

FIG. 21 is a representation of a block diagram of a DPR modulator,according to some embodiments of the present disclosure.

FIG. 22 is a block diagram of an encoder for DPR modulation, accordingto some embodiments of the present disclosure.

FIG. 23 is a block diagram for a waveform generator for DPR modulation,according to some embodiments of the present disclosure.

FIG. 24 is a block diagram of a symbol selector system module, which isused to select an appropriate symbol for use in DPR modulation,according to some embodiments of the present disclosure.

FIG. 25 is a plot of a camera sampling function, according to someembodiments of the present disclosure.

FIG. 26 is a plot of a modulated illumination function undergoing DPRmodulation at a frequency of 300 Hz, according to some embodiments ofthe present disclosure.

FIG. 27 is a plot of a convolution of a camera sampling function and aDPR modulated light signal, according to some embodiments of the presentdisclosure.

FIG. 28 is a model of the CMOS sampling function for a rolling shutter,according to some embodiments of the present disclosure.

FIG. 29 is a plot of a sampling function for a CMOS rolling shutter overmultiple frames, according to some embodiments of the presentdisclosure.

FIG. 30 is a high level flow chart of an algorithm for configuring amobile device to receive DPR modulated signals, according to someembodiments of the present disclosure.

FIG. 31 is a high level flow chart of an algorithm for minimizing andlocking camera settings using existing mobile device applicationprogramming interfaces (APIs), according to some embodiments of thepresent disclosure.

FIG. 32 is a high level flow chart of an algorithm for receiving DPRsignals on an image sensor, according to some embodiments of the presentdisclosure.

FIG. 33 is a high level flow chart of an algorithm for determining tonesembedded within a DPR illuminated area, according to some embodiments ofthe present disclosure.

FIG. 34 is a high level flow chart of an algorithm for performingbackground subtraction on images gathered from a DPR illuminated scene,according to some embodiments of the present disclosure.

FIG. 35 is a high level flow chart of an algorithm for performing motioncompensation on video frames when performing DPR demodulation, accordingto some embodiments of the present disclosure.

FIG. 36 is a photograph of a surface under illumination from DPRmodulated signals, according to some embodiments of the presentdisclosure.

FIG. 37 is a post-processed image of a DPR modulated scene afterperforming background subtraction, according to some embodiments of thepresent disclosure.

FIG. 38 is a post-processed image of a DPR modulated scene after rowaveraging, according to some embodiments of the present disclosure.

FIG. 39 is a plot of the 1-D spectral content of a DPR modulatedsurface, according to some embodiments of the present disclosure.

FIG. 40 is a plot of the 1-D spectral content of a DPR modulated surfaceafter removing DC bias, according to some embodiments of the presentdisclosure.

FIG. 41 is a 2-D FFT of a DPR modulated surface, according to someembodiments of the present disclosure.

FIG. 42 is a 2-D FFT of a DPR modulated surface after applying a lowpass filter, according to some embodiments of the present disclosure.

FIG. 43 is a 2-D FFT of a DPR modulated surface after applying a highpass filter, according to some embodiments of the present disclosure.

FIG. 44 is a representation of mobile devices receiving multiple sourcesof light, according to some embodiments of the present disclosure.

FIGS. 45 and 46 are block diagrams of light sources, according to someembodiments of the present disclosure.

FIG. 47 is a representation of mobile devices receiving multiple sourcesof light, according to some embodiments of the present disclosure.

FIG. 48 illustrates a user interface, according to some embodiments ofthe present disclosure.

FIG. 49 illustrates retrieval of a device profile, according to someembodiments of the present disclosure.

FIG. 50 shows a plot of variance in parameterized phase versus possiblemodulation frequencies, according to some embodiments of the presentdisclosure.

FIGS. 51-53 illustrate filtering of frequency components, according tosome embodiments of the present disclosure.

FIG. 54 illustrates a method for intelligently choosing the appropriatedemodulation, according to some embodiments of the present disclosure.

FIG. 55 illustrates a camera switching algorithm, according to someembodiments of the present disclosure.

FIG. 56 illustrates a method for resolving between multiple lightidentifiers, according to some embodiments of the present disclosure.

FIGS. 57-59 illustrates a mobile device receiving signals from multiplelights simultaneously, according to some embodiments of the presentdisclosure.

FIG. 60 is a block diagram of a DPR enclosure module, according to someembodiments of the present disclosure.

FIG. 61 is a representation of a self-identifying one-way opticaltransmitter, in accordance with some embodiments.

FIG. 62 is a representation of a mobile device receiving aself-identifying one-way optical transmission, in accordance with someembodiments.

FIGS. 63-66 are representations of self-identifying opticaltransmitters, in accordance with some embodiments.

FIG. 67 is a representation of a waveform for using pulse widthmodulation to transmit a signal, in accordance with some embodiments.

FIG. 68 is a representation of a high-brightness light sourcebroadcasting a modulated signal overhead, in accordance with someembodiments.

FIG. 69 is a representation of a one-way authentication system beingused in a checkout aisle, in accordance with some embodiments.

DESCRIPTION OF EXAMPLE EMBODIMENTS

The present disclosure relates to a serf-identifying one-way opticaltransmitter that broadcasts an optical signal via modulation of a lightsource. The transmitter is not necessarily designed for general-purposeillumination. Instead, the transmitter is designed to broadcast a signalthat can be successfully received if a mobile device user puts theirmobile device in close proximity with the transmitter. The userexperience for the receiver may be one in which the user “taps” theirdevice onto the transmitter to receive the ID code. Users tapping theirdevices in this way provides an advantage in terms of security, becauseusers come physically close to the transmitter in order to receive thesignal. A user's device coming in close proximity to the transmitteralso provides an improved user experience, because such an actionprovides tangible feedback. In some embodiments, the transmitter can actas a completely stand-alone device. The transmitter may be batterypowered and preloaded with a specific optical signal. In otherembodiments, the transmitter may be connected to a power and datanetwork if battery replacement is not viable for a particularapplication or the use of multiple signals is desired.

On the mobile device side, a camera-equipped mobile device may utilizeits camera to receive the transmitted signal. An application on themobile device may be invoked to initialize the camera and, once invoked,start to acquire images. These images can be captured in succession, inthe case of recording video, or can be captured individually. Theself-identifying one-way optical transmitter may be constantlybroadcasting the signal.

FIG. 1 represents a mobile device 103 receiving light 102 from a LEDlight source 101. The LED light source 101 may be any lighting sourceused for general purpose, spot illumination, or backlighting. The LEDlight source may come in several form factors but is not limited to:Edison screw in, tube style, large and small object backlighting, oraccent lighting spots and strips. For the purposes of this disclosure,any form of LED light is considered as a potential source capable oftransmitting information.

Light 102 is a modulated LED light source 101, and is part of thevisible electromagnetic wireless spectrum. LEDs are considered digitaldevices which may be rapidly switched on and off, to send signals abovethe rate that the human eye can see. This allows them to be exploited tosend digital data through the visible light itself. By modulating theLEDs, turning them on and off rapidly, one may send digital informationthat is unperceivable to the human eye, but is perceivable by applicablesensors, including but not limited to image sensors and other types ofphotosensors.

There are many modulation techniques used to send information throughlight 102. One technique, “On Off Keying” (OOK), is a scheme to transmitdigital data by rapidly switching a signal source on and off. OOK is thesimplest form of amplitude-shift keying (ASK) which is a modulationtechnique that represents digital data through either the presence orabsence of a carrier wave. When communicating with visible light, thecarrier wave takes the form of the transmitted light signal. Thereforeat a rudimentary level, when the light signal is turned “on” a digital“one” is perceived, and when the light signal is turned “off” a “zero”is perceived. Furthermore the rate at which the light signal is turnedon and off represents the modulation frequency. Note that regardless ofchanging the modulation frequency, the “carrier wave” remains unchangedas this is an inherent property of the light itself. For example thecarrier wave corresponding to a blue light signal is uniquely differentthan the carrier wave corresponding to a red light signal. While thesetwo signals differ only in the wavelength specific to their perceivedcolor, they can be perceived as two discrete signals.

In addition to OOK, another possible technique is defined as “DigitalPulse Recognition” (DPR). This modulation technique exploits the rollingshutter mechanism of a complementary metal-oxide-semiconductor (CMOS)image sensor. Due to their superior energy efficiency, CMOS sensors arepreferred to charge-coupled device (CCD) sensors on mobile devices. Whena CMOS image sensor with a rolling shutter takes an image, it does notexpose the entire image simultaneously. Instead, the rolling shutterpartially exposes different portions of the frame at different points intime. Typically, this causes various unwanted effects: skew, wobble, andpartial exposure. In the presence of an LED light driven by a pulsewidth modulated signal, images received from a CMOS sensor exhibit“residual banding” in the form of visible distortions. The image appearsto have alternating dark/white stripes. The stripes are a direct resultof the rolling shutter mechanism, and their width is proportional to thefrequency of the pulse width modulated (PWM) signal. Higher frequenciescorrespond to narrower stripes, and lower frequencies result in widerstripes. Practical frequency ranges for use with this technique arebetween 60 Hz and 5000 Hz. This technique allows one to exploit therolling shutter mechanism to recover digital data from an opticallyencoded signal.

DPR has the potential for much higher data rates than both OOK andfrequency shift keying (FSK). In FSK and OOK, the camera's frame ratelimits the data rate. The highest possible data rate is half of theframe rate, since each symbol spans over two frames. In DPR modulation,a single frame is sufficient for capturing the transmitted symbol.Furthermore, symbols are not “binary”—there are can be as many as 30different possibilities for a symbol.

In the DPR modulation scheme, image processing is used to measure thestripe width of the recorded image. By successively changing the LEDdriver frequency for each frame, information is essentially transmittedthrough recognition of the band widths. In the current design, 10separate frequencies are used. For a 30 frames per second (FPS) camera,this corresponded to an effective data transfer rate of ˜100 bits persecond (bps).

Both of these techniques are interesting because they can allow thetransmission of information through single color light sources, insteadof having to create lighting sources which contain multiple colorlights. In the world of LED lighting products, white light is majorlyachieved by layering a phosphorous coating on top of blue LEDs. Thecoating creates the visible perception of “white” light, instead ofblue. The alternative to this can be achieved through combining red,green, and blue LED lights; however this approach is expensive and powerinefficient as the lumens per watt properties differ between differentcolored LEDs. Blue LEDs are generally more energy efficient than theirred and green counterparts, which is why they are used in mostcommercial LED lighting products. It is because of this reason that itmakes the most sense to use a data modulation technique that uses asingle wavelength of light, rather than multiple, because this complieswith LED lighting products.

In addition to LED light sources, other types of light sources are alsocapable of transmitting information through modulation. Alternativeincandescent and fluorescent technologies can also be exploited toachieve data transmission, however the circuitry is more complex becausethe turn-on and turn-off times of incandescent and fluorescent lightsare subject to additional factors.

The modulation frequency of the light source is highly dependent on thereceiving circuitry. While incandescent and fluorescent technologiesgenerally do not “flicker” on and off during the course of normaloperation, LED lighting sources are sometimes designed to flicker abovethe rate which the eye can see in order to increase their longevity, andconsume less power. Most humans cannot see flicker above 60 Hz, but inrare instances can perceive flicker at 100 Hz to 110 Hz. To combat this,lighting manufacturers design flicker above 200 Hz into their lightingproducts.

Mobile device 103 may be a smart mobile device and is most commonlyfound in the form of mobile phones, tablets, and portable laptopcomputers. In order for a mobile device 103 to receive information 102from the LED light source 101 it has an embedded or attached sensorwhich is used to receive the incoming light 102 signals. One such sensoris a camera, which has a typical frame refresh rate between fifteen andsixty frames per second (fps). The fps is directly related to the speedat which optical signals can be transmitted and received by the camera.The sensor may capture a number of successive image frames that maylater be analyzed to determine if a light source is providinginformation through light.

Mobile device 103 may include a processor, module, memory, and sensor inorder to capture and analyze light received from light sources. Themobile device may analyze the successive image frames captured by thesensor by using the module. The module may be logic implemented in anycombination of hardware and software. The logic may be stored in memoryand run by processor to modify the successive images and analyze thesuccessive images to determine information encoded in the light of oneor more light sources. The module may be built in to the mobile deviceto provide the capabilities or it may be downloaded and installed. Themodule may be an application that runs on the mobile device whenselected by a user. The module may also be used to receive content andother information related to the position of the mobile device and toprovide this content to other modules or to the mobile device.

The reception of optically transmitted information is particularlyinteresting when used as an indoor positioning system. In a light-basedpositioning system, the physical locations of light sources may be usedto approximate the relative position of a mobile device 103 within lineof sight. On the mobile side, in addition to a receiving module, themobile device 103 may use information to determine position of themobile device. The mobile device may access a data source containinginformation about where the lights are physically located to determineposition. This data source may be stored locally, or in the case wherethe mobile device 103 has a network connection, the data source may bestored on an external server 703.

For scenarios where a network connection is not available, beforeentering an indoor space the mobile device 103 may optionally download a“map pack” containing the information used to locate itself indoors,instead of relying on an external server 703. In order to automate thisprocess, the mobile device 103 would first use an alternative existingtechnique for resolving its position and would use the gained locationinformation to download the appropriate map pack. The techniques forreceiving geo-location information include, for example, GPS, GSM, WiFi,user input, accelerometer, gyroscope, digital compass, barometer,Bluetooth, and cellular tower identification information. Thesetechniques may also be used to fill gaps between when a position of themobile device is determined using the light-based technique. Forexample, a mobile device may be placed at times so its camera does notcapture light sources. Between these times these alternative existingtechniques may be used for filling in position and location informationthat may be helpful to the user. The map pack would contain a map 902 ofthe indoor space the user is entering, locations of the lights from somesort of existing or third-party lighting plan 1103, and anylocation-dependent content 903 for the mobile device 103 to consume. Anyrequests for location information would simply access data storedlocally on the mobile device 103, and would not need to access a remoteserver via a network 601.

In terms of the experience when using a light-based positioning system,the indoor location reception and calculation may happen with little tono user input. The process operates as a background service, and readsfrom the receiving module without actually writing them to the displayscreen of the mobile device. This is analogous to the way WiFipositioning operates, signals are read in a background service withoutrequiring user interaction. The results of the received information maybe displayed in a number of ways, depending on the desired application.In the case of an indoor navigation application, the user would see anidentifying marker overlaid on a map of the indoor space they are movingaround in. In the case of content delivery, the user might see a mobilemedia, images, text, videos, or recorded audio, about the objects theyare standing in front of.

In scenarios where the mobile device 103 is in view of several lightsources, it may receive multiple signals at once. FIG. 2 is arepresentation of a mobile device 103 receiving identificationinformation 102 a-102 c from multiple LED light sources 101 a-101 c.Each light source is transmitting its own unique piece of information.In order to identify its position or receive location-based content, themobile device 103 may then use the received information to access adatabase 802 containing information about the relative positions of theLED light sources 101 a-101 c and any additional content 903. When threeor more sources of light are in view, relative indoor position may bedetermined in three dimensions. The position accuracy decreases withless than three sources of light, yet remains constant with three ormore sources. With the relative positions of lights 101 a-101 c known,the mobile device 103 may use photogrammetry to calculate its position,relative to the light sources.

Photogrammetry is a technique used to determine the geometric propertiesof objects found in photographic images. In the context of locatingmobile devices using light sources, photogrammetry refers to utilizingthe corresponding positions of LED light sources, and their positions in3-D space, to determine the relative position of a camera equippedmobile device. When three unique sources of light are seen by the cameraon a mobile device, three unique coordinates may be created from thevarious unique combinations of 101 a-101 c and their relative positionsin space can be determined.

For a mobile device 103 equipped with an image sensor the followingscenario may be considered. When multiple LED light sources appear inthe image sensors field of view, the sources appear brighter relative tothe other pixels on the image. Thresholds may then be applied to theimage to isolate the light sources. For example, pixel regions above thethreshold are set to the highest possible pixel value, and the pixelregions below the threshold are set to the minimum possible pixel value.This allows for additional image processing to be performed on theisolated light sources. The end result is a binary image containingwhite continuous “blobs” where LED light sources are detected, and darkelsewhere where the sources are not detected.

A blob detection algorithm may then be used to find separate LED lightsources. A minimum of three separate LED blobs are used to resolve the3-D position of a mobile device 103. Each LED blob represents a “regionof interest” for the information reception, and is simultaneouslytransmitting a unique piece of information via the modulated visiblesignal from the light source. For the purposes of reception, each regionof interest is processed independently of other regions of interest andis considered to be uniquely identifiable. A center of mass calculationfor each region may be performed to determine the pixel coordinates ofthe center of each LED light source. This center of mass calculation isperformed for each frame to track the regions of interest as they movearound the image.

Once the regions of interest are established, a detection algorithmcaptures multiple image frames for each region of interest in order toreceive the visible light signal contained in each blob. For each framein a detected region of interest, a threshold algorithm determineswhether the frame contains a “1” (in the case of an aggregate pixelvalue above the threshold), or a “0” (in the case of an aggregate pixelvalue lower than the threshold). The threshold algorithm is used sincethe communication is asynchronous, so the camera receiver period mayoverlap between the transmission of a “1” and a “0” from the LED lightsource.

The result of converting successive image frames in a region of interestto binary values is in essence a down-sampled digital version of thesignal received from the LED light source. Next, demodulation of thedown-sampled digital signal is used to recover the transmitted bits.This down sampling is used due to the fact that the signal modulationfrequency should be above the rate at which the human eye can see, andthe image sensor frame rate is typically limited to 15-30 fps.

At a lower level, the mobile device 103 processes data on aframe-by-frame basis. Each frame is split into separate regions ofinterest, based on the detection of light sources. For each region ofinterest, a thresholding algorithm is used to determine whether a givenregion is “on” or “off”. This is done by taking the average pixel valuefor the region and comparing it to the threshold value. If the region is“on”, the demodulator assumes the light source has just transmitted a“1”. If the region is “off”, the demodulator assumes the light sourcehas sent a “0”. The result of this is the equivalent of a 1-bitanalog-to-digital conversion (ADC), at a sampling rate which is equal tothe frame rate of the camera.

After a frame is processed, the results of the ADC conversation arestored in a circular buffer. A sliding correlator is applied to thebuffer to look for the presence of start bits 402. If start bits 402 arefound, the demodulation algorithm assumes it is reading a valid packetof information 401 and proceeds to capture the rest of the transmission.Two samples are used for each bit, so the algorithm creates a linearbuffer that is twice the size of the remaining packet. Each subsequentADC is written sequentially to the linear buffer. When the linear bufferis filled, the demodulation algorithm performs a Fast Fourier Transform(FFT) on the buffer to recover the transmitted signal.

FIG. 3 describes internal components commonly found in LED light source101 with the addition components to allow for the transmission ofoptical signals. The LED light source 101 typically contains analternating current (AC) electrical connection 301 where it connects toan external power source, an alternating current to direct current(AC/DC) converter 302 which converts the AC signal from the power sourceinto an appropriate DC signal, a modulator 304 which interrupts power tothe LEDs in order to turn them on and off, a microcontroller 305 whichcontrols the rate at which the LEDs are modulated, and a LED drivercircuit 303 which provides the appropriate amount of voltage and currentto the LEDs.

Electrical connection 301 is an electrical source that is used to supplypower to the LED light source 101. This most commonly comes in the formof a 120 Volt 60 Hz signal in the United States, and 230 Volt 50 Hz inEurope. While depicted in FIG. 3 as a three pronged outlet, it may alsotake the form of a two terminal Edison socket which the bulb is screwedinto, or a bundle of wires containing a live, neutral, and/or ground.When considering other forms of lighting such as backlighting and accentlighting, the electrical connection may also come in the form of a DCsource instead of an AC source.

Most LED light sources contain an AC/DC converter 302 that converts thealternating current from the power source 301 to a direct current sourceused internally by the components found inside the bulb or light source.The converter takes the alternating current source commonly found inexisting lighting wiring and converts it to a direct current source. LEDlight sources generally use direct current, therefore an AC/DC converteris found in most lighting products regardless of form factor.

LED driver 303 provides the correct amount of current and voltage to theLEDs contained inside the lighting source. This component is commonlyavailable and may have either a constant-current or constant-voltageoutput. The LEDs found inside most lighting sources arecurrent-controlled devices, which require a specific amount of currentin order to operate as designed. This is important for commerciallighting products because LEDs change color and luminosity in regards todifferent currents. In order to compensate for this, the LED drivercircuitry is designed to emit a constant amount of current while varyingthe voltage to appropriately compensate for the voltage drops acrosseach LED. Alternatively, there are some high voltage LEDs which requirea constant voltage to maintain their color and luminosity. For thesecases the LED driver circuitry provides a constant voltage while varyingthe current.

Modulator 304 serves the function of modulating the LED light source 101on and off to optically send light 102 signals. The circuits featuringthe modulator may simply consist essentially of solid-state transistorscontrolled by a digital input. In essence, the modulator 304 turns theLEDs on and off by allowing or preventing current flow. When currentflows through the modulator with the switches closed the LEDs turn on,and when the switches are open in the modulator no current can flow andthe LEDs turn off. When the modulator is controlled by an additionallogic component, it has the ability to send repeating patterns of on/offsignals in order to transmit digital data through the visible light 102.The modulator interfaces directly in between the AC/DC converter 302 andthe LED driver 303, and is controlled by a microcontroller 305.

The microcontroller 305 provides the digital input signal to themodulator unit 304. This function may also be achieved using afield-programmable gate array (FPGA), but typically consumes more powerwith added complexity. The microcontroller's 305 task is to send apre-determined sequence of signals to the modulator 304 which theninterfaces with the LED driver 303 to modulate the outgoing visiblelight from the LED source 101. The microcontroller contains anonvolatile memory storage area, which stores the identification code ofthe light signal. Examples of possible nonvolatile memory sourcesinclude programmable read only memory (PROM), electrically erasableprogrammable read only memory (EEPROM), or Flash.

In regards to the microcontroller pins, the microcontroller 305 containsa digital output pin, which is used to modulate the light output. Togenerate the output signal waveforms, timer modules within themicrocontroller 305 are used. Typical logic levels for the digitaloutput are 3.3V and 5V. This digital output feeds into the modulator 304which interrupts the driver circuit 303 for the LED light source 101.Alternatively, if the LED light source requires lower power, such asbacklighting or individual LED diodes, the output of the microcontroller305 could also be used to drive the light sources directly.

The sequence of signals sent from the microcontroller 305 determines theinformation that is transmitted from the LED light source 101. FIG. 4describes the information 401 format of the optically transmittedinformation from the light 102. At the highest level, each packet ofinformation contains some sort of starting bit sequence, which indicatesthe beginning of a packet, followed by data 403, and some sort of errordetection identifier. The size and position of each portion ofinformation is dependent on the application and is also constrained byrequirements of the receiving device.

Each packet of information 401 transmitted from the LED light source 101contains a sequence of starting bits 402, followed by data 403, and thenterminated with an error detection code 404. Since the LED light sources101 are continually broadcasting information 401, erroneous packets aresimply discarded while the receiver listens for the starting bits 402,indicating the beginning of the next packet. In cases where multiplesources of light are observed by a mobile device 103, multiple pieces ofinformation 401 are received simultaneously.

Information 401 describes the encoded information that is transmitted bythe LED light source 101. The information 401 is contained in a packetstructure with multiple bits which correspond to numeric integer values.The data 403 portion of the information packet may include unique IDcodes 701. Currently the data 403 size is set to 10 bits, but may be ofvarying length. Each bit represents a binary “1” or “0”, with 10 bits ofdata 103 corresponding to 1024 possible values. This corresponds to 1024unique possibilities of ID codes 701 before there is a duplicate. The IDcode may include location information in the ID code that provides ageneral indication of geographical location of the light. Thisgeographical location information may be used to more quickly locatelight source information that is used in determining indoor positioningon the mobile device. For example, the geographical information maypoint to a database to begin searching to find relevant information forpositioning. The geographical information may include existing locationidentifiers such as area code, zip code, census tract, or any othercustomized information.

The ID code 701 is static and is assigned during the calibration phaseof the LED light source 101 during the manufacturing process. One methodto assign the ID code 701 is to place instructions to generate a randomcode in the nonvolatile memory. Once the LED light source 101 is poweredon the microcontroller reads the ID code 701 from the nonvolatile memorystorage area, and then uses this code for broadcasting each and everytime it is subsequently powered on. Since the ID code 701 is static,once it is assigned it will be forever associated locally to thespecific LED light source 101 which contains the microcontroller 305.

FIG. 5 describes the components found in mobile devices 103 that arecapable of receiving optical information. At the highest level themobile device contains an image sensor 501 to capture opticallytransmitted information, a central processing unit 502 to decipher andmanage received information, and a network adapter 503 to send andreceive information.

Photosensors are devices which receive incoming electromagnetic signals,such as light 102, and convert them to electrical signals. In a similarfashion, image sensors are arrays of photosensors that convert opticalimages into electronic signals. The ability to receive signals frommultiple sources is an important benefit when using image sensors forreceiving multiple optical signals.

Image sensor 501 is a typical sensor which is found in most smartdevices. The image sensor converts the incoming optical signal into anelectronic signal. Many devices contain complementarymetal-oxide-semiconductor (CMOS) image sensors; however, some still usecharge-coupled devices (CCD). CMOS image sensors are the more popularchoice for mobile devices due to lower manufacturing costs and lowerpower consumption. There are several tradeoffs to consider when choosingan image sensor to perform photogrammetry on multiple LED light sources101. One tradeoff is between the camera resolution and the accuracy ofthe photogrammetric process when triangulating between multiple lightsources—increasing the number of pixels will increase the accuracy.There is also another tradeoff between the data rate of the transmissionand the sampling rate (in frames per second) of the camera. The datarate (in bits/second) is half the frame rate of the camera (e.g., a 30fps camera will receive 15 bps). And finally when determining the lengthof the information 401 packet, the larger the size the longer thereception period, as more bits generally requires longer samplingperiods to capture the full message.

CPU 502 is typically a generic CPU block found in most smart devices.The CPU 502 is in charge of processing received information and sendingrelevant information to the network adapter 503. Additionally the CPUhas the ability to read and write information to embedded storage 504within the mobile device 103. The CPU 502 may use any standard computerarchitecture. Common architectures for microcontroller devices includeARM and x86.

The network adapter 503 is the networking interface that allows themobile device 103 to connect to cellular and WiFi networks. The networkconnection is used in order for the mobile device 103 to access a datasource containing light ID codes 701 with their corresponding locationdata 702. This may be accomplished without a data connection by storinglocation data 702 locally to the mobile device's 103 internal storage504, but the presence of a network adapter 503 allows for greaterflexibility and decreases the resources needed. Furthermore, the networkadapter 503 is also used to deliver location dependent content to themobile device when it is connected to a larger network 601.

FIG. 6 is a representation of multiple LED sources sending light 102 a-dcontaining identification information 102 to multiple mobile devices 103a-103 b. In this instance the light sources are acting as non-networkedbroadcast beacons; there are no networking modules or physical datawires connecting them. This property is desirable when looking towards acommercial installation of numerous LED light sources 103 a-103 b, asadditional wiring and networking will not be required. However, in orderto receive relevant information the mobile devices have the ability tosend and receive additional information from a local source or a network601. Once the mobile device 103 receives identification information 401from the light sources, it then asks a local or remote source foradditional information.

Enclosed area 602 is a spatial representation of an enclosed roomcontaining four LED sources 101 a-101 d and two mobile devices 103 a-103b, meaning that they may operate next to each other withoutinterference. As a rule of thumb if the received image feed from themobile device sees one or more distinct bright sources of light, it hasthe ability to differentiate and receive the unique information withoutinterference. Because the light capture is based on line of sight,interference is mitigated. In this line of sight environment,interference may arise when the light capture mechanism of the mobiledevice is blocked from the line of sight view of the light source.

Network 601 represents a data network that may be accessed by mobiledevices 103 a-103 b via their embedded network adapters 503. The networkmay consist of a wired or wireless local area network (LAN), with amethod to access a larger wide area network (WAN), or a cellular datanetwork (Edge, 3G, 4G, LTS, etc). The network connection provides theability for the mobile devices 103 a-103 b to send and receiveinformation from additional sources, whether locally or remotely.

FIG. 7 describes how the mobile device 103 receives location data 702.In essence, the mobile device 103 sends decoded ID codes 701 through anetwork 601 to a server 703, which sends back location information 702.The decoded ID codes 701 are found in the information 401, which iscontained in the optically transmitted signal. After receiving thissignal containing a unique ID code 701 the mobile device 103 sends arequest for location data 702 to the server 703, which sends back theappropriate responses. Additionally the request could include othersensor data such as but not limited to GPS coordinates andaccelerometer/gyroscope data, for choosing between different types oflocation data 702 and any additional information.

Location data 702 is the indoor location information which matches thereceived information 401. The location data 702 corresponds to indoorcoordinates which match the ID code 701, similar to how outdoor GPS tagsknown locations of interest with corresponding information. The locationdata 702 could also contain generic data associated with the lightidentification information 401. This could include multimedia content,examples of which include recorded audio, videos, and images. Thelocation data 702 may also vary depending, for example, on othercriteria such as temporal criteria, historical criteria, oruser-specified criteria.

The temporal criteria may include the time of day. The historicalcriteria may include user location history (e.g., locations visitedfrequently), Internet browsing history, retail purchases, or any otherrecorded information about a mobile device user. The user-specifiedcriteria may include policies or rules setup by a user to specify thetype of content they wish to receive or actions the mobile device shouldtake based on location information. For example, the user-specifiedcriteria may include how the mobile device behaves when the user isclose to an item that is on sale. The user may specify that a coupon ispresented to the user, or information about the item is presented on themobile device. The information about the item may include videos,pictures, text, audio, and/or a combination of these that describe orrelate to the item. The item may be something that is for sale, adisplay, a museum piece, or any other physical object.

Server 703 handles incoming ID codes 701, and appropriately returnsindoor location data 702 to the mobile devices 103. The handling mayinclude receiving incoming ID codes, searching databases to determinematches, calculating position coordinates based on the ID codes, andcommunicating indoor location data 702. Since the LED light sources 101are acting as “dumb” one-way communication beacons, it is up to otherdevices to determine how to use the ID codes to calculate positioninformation and deliver related content. In some embodiments, the server703 may include the information used to link ID codes 701 to physicalspaces and to deliver location-specific content. The server is designedto handle the incoming requests in a scalable manner, and return resultsto the mobile devices in real-time.

The server may include one or more interfaces to the network that areconfigured to send and receive messages and information in a number ofprotocols such as Internet Protocol (IP) and Transmission ControlProtocol (TCP). The protocols may be arranged in a stack that is used tocommunicate over network 601 to mobile device 103. The server may alsoinclude memory that is configured to store databases and informationused in providing position coordinates and related location basedcontent. The server may include one or more modules that may beimplemented in software or other logic. These modules may performcalculations and perform operations to implement functionality on theserver. The server may use one or more processors to run the modules toperform logical operations.

To describe the server interaction in more detail, FIG. 8 delves intolocation-specific areas 801 containing databases 802 and web services803. The areas 801 represent a subset of databases 802 and web services803 for individual locations where there are installed LED light sources101. The server 703 directly communicates with these installations,which have their own separate sets of information. At a high level,databases 802 represent the stored information pertaining to a specificarea 801, while the web services 803 represent services which allowusers, customers, administrators, and developers access to the ID codes,indoor locations, and other information.

In order to send relevant information, after each received ID code 701,the server 703 requests information pertaining to the specific area 801.Contained in each area 801, are databases which contain informationcorresponding to the specific ID code 701. This information can takemultiple formats, and has the ability to be content specific to avariety of static and dynamic parameters.

In order to optimize response time, the server 703 may constrain itssearch space by using existing positioning technologies available to themobile device 103 or from information in the light source ID codedepending on the embodiment. In essence the server looks for the lightIDs 901 within a specific radius of the current approximate position ofthe mobile device 103, and ignores those that are geographicallyirrelevant. This practice is known as “geo-fencing”, and dramaticallyreduces the request/response time of the server 703. As finalverification, if the database 802 contains one or more of the same IDswithin the current search space that match the ID codes received by themobile device 103 within a specific time frame, then a successfultransaction can be assumed.

As seen in FIG. 9, each database 802 contains numerous sub-categorieswhich store specific types of information. The categories are labeledlight IDs 901, maps 902, content 903, and analytics 904.

Light IDs 901 is a category which contains records of the individuallight ID codes 701 which are contained in an area 801. In a typicallight positioning enabled installation, there will be tens to hundredsof unique LED light sources 101 broadcasting unique ID codes 701. Thepurpose of the light IDs 901 database is to maintain and keep a recordof where the ID codes 701 are physically located in the area 801. Theserecords may come in the form of but are not limited to GPS (latitude,longitude, and altitude) coordinates that are directly mapped into anindoor space. For instance, most indoor facilities have informationabout the number of installed lights, how far apart they are spaced, andhow high the ceilings are. This information may be matched with buildingfloor plans or satellite imagery to create a digital mapping of whereeach light is positioned.

To expand upon the Light IDs 901 category, additional information maycome in the form of location-specific maps 902. These maps may take onmany physical and digital forms, either directly from the management ofthe location, or a third-party vendor or outside source. In addition tomapping information, location-specific content 903 and analytics 904 arealso contained inside the databases 802.

FIG. 10 is a description of the ID log 1001 information contained in theLight IDs database 901. It is a representation of the file structurethat contains individual records corresponding to individual light IDcodes 701 found within different areas 801. In a typical area 801 thereis a possibility of having duplicate ID codes 701 since there are afinite number of available codes. The size of the ID code 701 isproportional to the length of the data 403 field contained in theoptical information 401.

To deal with duplicate ID codes 701, additional distinguishinginformation may be contained inside of the individual log records; ID 11001, ID 2 1003, and ID 3 1004. This information may contain additionalrecords about neighboring ID codes 701 that are in physical proximity ofthe LED light source 101, or additional sensor data including but notlimited to: accelerometer or gyroscope data, WiFi triangulation orfingerprinting data, GSM signature data, infrared or Bluetooth data, andultrasonic audio data. Each additional sensor is an input into aBayesian model that maintains an estimation of the current smartphoneposition and the uncertainty associated with the current estimation.Bayesian inference is a statistical method used to calculate degrees ofprobability due to changes in sensory input. In general, greater numbersof sensory inputs correlate with lower uncertainty.

In order to calibrate the light-based positioning system, a userequipped with a specific mobile application (app) will need to walkaround the specific area 801. The mobile application contains map 902information of the indoor space, with the positions of the LED lightsources 101 overlaid on the map. As the user walks around, they willreceive ID codes 701 from the lights. When the user receives an ID code701, they will use the map on the mobile app to select which LED lightsource 101 they are under. After the user confirms the selection of thelight, the mobile application sends a request to the server 703 toupdate the light location contained in the lighting plan 1103 with theID code 701. Additional user-provided 1104 metadata including but notlimited to current WiFi access points, RSSI, and cellular towerinformation may also be included with the server request to updateadditional databases.

In addition to manual calibration, calibration of LED light source 101locations may also be achieved via crowd-sourcing. In this algorithm, asmobile application users move around an indoor space receiving ID codes701, they will send requests to the server 703 containing the light IDcode 701 received, the current approximate position (based on otherpositioning techniques such as WiFi, GPS, GSM, and inertial sensors) andthe error of the current approximation. Given enough users, machinelearning algorithms on the server 703 may be used to infer the relativeposition of each LED light source 101. The accuracy of this calibrationmethod depends heavily on the number of mobile application users.

FIG. 11 is a description of the maps database 902 and map log 1101information containing floor plans 1102, lighting plans 1103,user-provided information 1104, and aggregated data 1105. Map log 1101is a representation of the file structure that contains the informationfound inside the maps database 902. Information may come in the form ofbut is not limited to computer-aided drafting files, user-providedcomputerized or hand drawn images, or portable document formats. Theinformation residing in the maps database 902 may be used both tocalibrate systems of multiple LED light sources 101, and to augment thelocation data 702 that is sent to mobile devices 103.

Floor plan 1102 contains information about the floor plan for specificareas 801. The contained information may be in the form ofcomputer-aided drafting files, scanned images, and legacy documentspertaining to old floor plans. The information is used to build a modelcorresponding to the most recent building structure and layout. Thesemodels are subject to changes and updates through methods including butnot limited to crowd sourcing models where users update inaccuracies,third-party mapping software updates, and additional input from privatevendors.

Lighting plan 1103 contains information about the physical lightingfixture layout, electrical wiring, and any additional informationregarding the lighting systems in the area 801. This information mayalso come in a variety of physical and digital forms such as the floorplan 1102 information. The lighting plan 1103 information is used in thecalibration process of assigning light ID codes 701 to physicalcoordinates within an area 801. In essence, a location with multiple LEDlight sources 101 acts as a large mesh network except, in this case,each node (light ID 701) is a non-networked beacon of information thatdoes not know about its surrounding neighbors. To help make sense ofmultiple light ID codes 701, the lighting plan 1103 information is usedas one of many ways to tell the backend server 703 where LED lightsources 101 are located.

User-provided information 1104 contains additional data that the usermanually uploads in regards to building changes, updates, or newinformation that is acquired. The user in this case is most likely thefacility manager or staff member, but such information may alsooriginate from an end user of the system who contributes via a crowdsourcing or machine learning mechanism. For instance, if an end user wasusing a light-based positioning system in a museum and was unable tofind a particular exhibit or noticed inaccurate information in regardsto location or classification of the exhibit, they could red flag theoccurrence using their mobile device 103. When coupled with data fromadditional users, sometimes known as a crowd-sourcing method, thisuser-provided information 1104 may be used to update and repairinaccuracies in the maps 902 database.

Aggregated data 1105 contains information that is gathered by the systemthat may be used to augment the current information that is known aboutthe mapping environment. This may occur during normal operation of thesystem where multiple mobile devices 103 are constantly sending andreceiving location data 702 from the server 703. Over time theaggregation of this data may be used to better approximate how light IDcodes 701 correspond to the physical locations of the LED light sources101. For instance, if multiple mobile devices 103 consistently receive anew ID code 701, in a repeatable pattern with respect to additionalknown ID codes 701 and other sources of location information, then thisinformation may be recorded and stored in the aggregated data 1105database. This information may additionally be used to recalibrate andin essence “self-heal” a light-based positioning system.

FIG. 12 is a description of the content database 903 and content log1201 information containing static content 1202, user-based content1203, and dynamic content 1204. Content log 1201 is a representation ofthe file structure that contains the information found inside thecontent database 903. Static content 1202 refers to unchanginginformation that is associated with the specific area 801. This mayrefer to the previous example in which a facility manger loads specificcontent into the content 903 database before a user enters the specificarea 801. This type of information may take the form of but is notlimited to audio recordings, streaming or stored video files, images, orlinks to local or remote websites.

User-based content 1203 refers to content that is dependent on usercriteria. The content may depend on, but is not limited to, user age,sex, preference, habits, etc. For instance, a male user might receivedifferent advertisements and promotions than a female would.Additionally, age and past purchase habits could also be used todistinguish which is the correct piece of content to be presented to theuser.

Dynamic content 1204 refers to content which changes with varyingfrequency. The content may change dependent on a temporal bases, daily,weekly, monthly, etc. For instance, seasonal marketing and content couldbe automatically presented to the user dependent on the month of theyear, or content in the form of morning, evening, or nightly specialscould be presented numerous times throughout the individual day.

In addition to content, point of purchase 1205 information may bedelivered as well. This could be implemented by using the received IDcode 701 to a secure connection that establishes and completes atransaction linked to a user's selected payment method. Additionally, astandalone point of purchase feature could be implemented by simplylinking ID codes 701 directly to merchandise or services.

FIG. 13 is a description of the analytics database 904 and analytics log1301 information containing frequency 1302, dwell time 1303, path taken1304, and miscellaneous 1305. Analytics log 1101 is the file structurethat contains the information found inside the analytics database 904.Frequency 1302 refers to the number of times each end user visits aparticular location inside of a specific area 801. Separate records aremaintained for individual users, and the frequency is aggregated andsorted in the frequency files database 904.

Dwell time 1303 refers to the time spent in each particular locationinside a specific area 801. Separate records are maintained forindividual users, and the dwell times are aggregated and sorted in thedwell time file. Path taken 1304 refers to the physical path taken by auser in each specific area 801.

Consider an example that combines many of the above descriptions,involving a store owner that installed a light-based indoor positioningsystem and a customer walking around the store using a mobile device 103capable of receiving optically transmitted information. The customerdrives to the parking lot of the store, parks, and walks in. Using thebackground sensors and location services available to her phone asmodeled in FIG. 16, the customer's mobile device 103 already knows thatshe has approached, and most likely entered a store outfitted with alight-based positioning system. Once this information is known, theapplication running on the customer's mobile device 103 initiatesseveral background services and begins to start looking for opticalsignals as depicted in FIG. 15.

Prior to the customer entering the store, the store owner has alreadycalibrated and preloaded the database 802 with the unique LED lightsources 101, map 902 information pertaining to the store floor plan1102, user-provided 1104 product locations, and content 903 in the formof multimedia and local deals in the form of promotions that may only beactivated by visiting that particular section of the store.

In the meantime, the customer is walking around the store looking tofind particular items on her shopping list that she has alreadydigitally loaded onto her mobile device 103. Next, the customer isprompted by her mobile device 103 that one of the items on her list haschanged locations and an image of the store layout is displayed with aflashing icon indicating where her desired product has moved. The mobilephone may guide her to the new product. Then as soon as she gets closeto the product, an informational video is prompted on her screendetailing the most popular recipe incorporating that product and how itis prepared. Finally, in addition to finding her desired product, thecustomer receives a discount promotion for taking the time to seek outthe new location of the product.

In addition to the services offered by this system to the customer, thestore owner now gains value from learning about the shopping experiencesof the customer. This comes in the form of aggregated data that iscaptured and stored in the analytics 904 section of his store's database802. This example is one of many applications that may be enabled withan accurate indoor light-based positioning system.

FIG. 14 is a process describing the act of receiving location andcontent information through visible light. User places mobile deviceunder light 1401 corresponds to the act of physically placing a cameraequipped mobile device 103 underneath an enabled LED light source 101.The user stands approximately underneath or adjacent the LED lightsource 101, and the mobile device has the LED light source 101 in viewof the camera lens.

The next block, sample image sensor 1402, refers to the act of turningon and reading data from the embedded image sensor in the mobile device103. Receive ID? 1403 is a decision block which either moves forward ifa location ID is received, or returns to sample the image sensor 1402.Get location data corresponding to ID from server 1404 occurs once alocation ID has been received. The mobile device queries the serverasking for location data 702 relevant to the ID code. This describes theprocess of a user obtaining an ID code 701 from a non-networked LEDlight source 101, and using the unique identifier to look up additionalinformation from either the server 703 or a locally stored source.

Finally, Content? 1405 is another decision block which determines ifthere is location-based content associated with the received ID code. Ifcontent is available the process continues on to the last block 1406where the content is queried; if not, the process ends. As describedabove, the get content data corresponding to ID from server 1405 refersto the act of retrieving content data associated with a known locationfrom either a server 703 or local source.

FIG. 15 is a process describing the act of turning on the applicationbackground services and determining when to sample the image sensor.Initiate background service 1 1501 is the primary background runningservice on the mobile device. This service is tasked with initiating afunction that can communicate wirelessly to determine if the mobiledevice is close to an enabled area. The wireless communication includesradio frequency communication techniques such as global position system(GPS), cellular communication (e.g., LTE, CDMA, UMTS, GSM), or WiFicommunications. Determine position 1502 is the function thatperiodically samples the wireless communication signal and based ondistance parameters decides whether or not the mobile device is closeenough to an area to move forward to the next service.

Light positioning enabled? 1503 is a decision block that moves forwardif the mobile device is close to an enabled location, or repeats theprevious function if not. Initiate background service 2 1504 isactivated once the mobile device enters an enabled area. The service istasked with initiating the functions that receive location informationvia the modulated light.

Sample ambient light sensor 1505 is the first function of the previousservice that samples the ambient light sensor data as soon as the sensordetects a change. The function of this task is to determine if thesensor has gone from dark to light, if the user takes the device out ofa pocket or enclosure, or from light to dark, the user has placed thedevice inside of a pocket or enclosure. As an alternative to samplingthe light sensor, the algorithm could also look for a change in theaccelerometer reading. This may correspond to the user taking the phoneout of their pocket. Detect change? 1506 is the decision block thatmoves forward if the ambient light sensor has gone from dark to light,meaning that the mobile device is potentially in view of surroundingmodulated light.

FIG. 16 is a process describing the act of determining a mobile device'sposition using a variety of information sources. Sample GPS/GSM 1601refers to the act of determining if the mobile device is close to anenabled area. Enabled area? 1602 is a decision block which moves forwardif the mobile device is close to a enabled area, or returns to theprevious block if not.

Sample alternative sources 1603 refers to the act of leveraging existingalternative positioning technologies such as WiFi, Bluetooth,ultrasound, inertial navigation, or employing an existing service usingone or more of any available services. Record internal sensor data 1606is a task which records the current accelerometer data for a period oftime before returning to the Sample image sensor 1402 block. This taskis performed so that location information is constantly being collectedeven when modulated light is not being detected. This allows the mobiledevice and/or server to keep track of the mobile device's position.

FIG. 17 is a system diagram describing how a client device 1704interacts with a light-based positioning system 1709. Network 601 is ageneric local or remote network used to connect mobile devices 103contained in locations A 1701, B 1702, and C 1703 with the light-basedpositioning service 1709.

Each location contains multiple LED light sources 101, each of whichbroadcast unique identification codes 701. In order to interact with thesystem from an operator's perspective, a mobile device may use thedatabase service application 1710 which contains multiple privilegelevels for different levels of access. The client privilege leveldetermines read/write permissions to each of these databases. Theselevels include users 1705 which refer to general front end system users,administrators 1706 which are usually IT or operations management levelwithin an installation, developers 1707 which have access to theapplication programming interfaces of the system for use in customapplication development, and root 1708 level which contains mastercontrol over the users and access to everything contained in the systemand databases.

Mobile devices in each location 1701, 1702, and 1703 receiveidentification codes 701 from lights in their respective locations. Theythen send the received identification codes 701 through the network 601which connects to database service application 1710, through userapplication 1705, and has read access to maps 902 and content, and writeaccess to analytics 904. A generic client, 1704, connects to databaseservice application 1710 through network connection 601.

The client uses a password authorized login screen to access therespective permission status. Clients with administrator permissionshave read/write access to light IDs 901, read access to maps 902,read/write access to content 903, and read access to analytics 904.Clients with developer permissions 1707 have read access to light IDs,read access to maps 902, read/write access to content 903, and readaccess to analytics 904. A client with root permissions 1708 hasread/write access to databases 901-904.

As an overview, FIG. 17 describes the top-down approach to an exemplaryimplementation of a light-based positioning system. At the highestlevel, known locations of installed non-network standalone LED lightsources 101 are used to accurately identify the relative position ofmobile devices 103. In order to obtain identification information fromthe lights, the background processes running on the mobile device 103have been described in FIGS. 14, 15, and 16. Once the mobile device hasacquired a unique or semi-unique ID code 701 from the light orcombination of lights, it uses this information to query a database 802for additional information. This information may come in many forms, andis used to create a more personalized experience for the user. Asinitially mentioned, this local experience is used for location-awaremobile computing, and augmented reality applications. In addition tolocal personalized information, location-based analytics applicationsmay be enabled from the aggregated data and traffic running through theserver 703.

The use of light-based positioning capabilities provide a number ofbenefits. For example, the positioning information obtained by usinglight sources is highly precise compared to alternative techniques forpositioning information. The accuracy of a light-based positioningsystem may be down to a few centimeters in three dimensions in someembodiments. This positioning ability enables a number of usefulservices to be provided. In certain embodiments, additional mobiledevice information may be used in combination with the positioninginformation. For example, accelerometer position information may be usedin conjunction with light source based position to offer augmentedreality or location aware content that relevant to the device'sposition. The relevant content may be displayed to augment what is beingdisplayed on the mobile device or the display can provide relevantinformation. Applications on the mobile device may also be launched whenthe mobile device enters certain areas or based on a combination ofcriteria and position information. The applications may be used toprovide additional information to the user of the mobile device.

The light-based positioning systems and techniques may also be used tomanage and run a business. For example, the light-based positioning mayhelp keep track of inventory and to make changes to related databases ofinformation. In a warehouse, for example, the light-positioning systemmay direct a person to where a particular item is located by givingdirections and visual aids. The light positioning may even providepositioning information to direct the person to the correct shelf theitem is currently residing on. If the person removes the item, themobile device may update the inventory databases to reflect the change.The same function may be implemented in a store environment asmerchandise locations are changed or updated. This information may thenbe used in providing content to a user. For example, if a shopper wantsmore information about an item, the updated location may be used tolocate the item or direct the shopper to an online website to purchasean out-of-stock item. In some embodiments, the mobile device using thelight-based positioning technique in conjunction with a wirelessconnection and other information may be used to provide non-intrusivedata collection on customers. The data collection of how customers movethrough a store and where they spend time may be used to improve layoutof stores and displays of merchandise.

The light-based positioning systems are also easy and low-cost to set upcompared to other location-positioning systems. Since each light sourceoperates autonomously, a building owner only needs to swap out existinglight sources for those that provide light-based information to acamera-enabled device. The light sources are non-networked independentbeacons that broadcast identification codes configured whenmanufactured. This allows the light sources to be manufactured at alower cost compared to networked light sources. Further, thenon-networked independent beacon light sources in the light-basedpositioning system may be easier for building owners to install.

The light-based positioning system may also include optimizations insome embodiments. For example, location information obtained from eitherthe identification code or from alternative techniques can be used toreduce latency in determining position information. This optimizationmay work through geo-fencing by constraining the search area to findinformation regarding the captured light sources more quickly. This canreduce the overall delay experienced by a user from the time the mobiledevice captures the light sources to when relevant position informationis provide to the mobile device and/or relevant content is provided tothe mobile device.

Efficient Light Bulbs for DPR Schemes

One of the biggest challenges facing beacon-based light-positioningsystems is managing the additional power consumption ofcommunication-enabled lighting devices in comparison to that ofnon-communicating devices. Lighting sources 101 in general, regardlessof form factor or technology, are differentiated in part by their powerconsumption; generally, the less the better. Accordingly, higher energyefficiency is one of the core economic forces driving adoption ofLight-Emitting-Diodes (LEDs). However, when using light sources 101 as ameans for communication devices, the power requirements tend to increasedepending on the modulation scheme since energy must be divided betweenthe carrier wave and the modulation wave. There are many differenttechniques for transmitting data through light, for example, asdiscussed in U.S. Ser. No. 12/412,515 and U.S. Ser. No. 11/998,286, andU.S. Ser. No. 11/591,677, the entire disclosure of each of which isincorporated by reference herein. However, these techniques haveprimarily been pursued without considering their impact on light source101 parameters, including efficacy, lifetime, and brightness. Sincelight sources 101 are first and foremost illumination devices, and notcommunication devices, the communication function takes a secondaryrole. The present disclosure utilizes Digital Pulse Recognition (DPR)modulation as a technique for transmitting data while minimizing theimpact on illumination devices.

FIGS. 18A-C represent several digitally modulated light sources 101 a-cwith varying duty cycles; a low duty cycle 1801, a medium duty cycle1802, and a high duty cycle 1803. A duty cycle is a property of adigital signal that represents the proportion of time the signal spendsin an active, or “on,” state as opposed to an inactive, or “off,” state.A light source with a low duty cycle 1801 is inactive for a highproportion of time. A light source with a medium duty cycle 1802 isinactive for about the same proportion of time that it is active. Alight source with a high duty cycle 1803 is active for a high proportionof time. The duty cycle of a light source affects the luminosity of thelight source. A light source having a higher duty cycle generallyprovides more luminosity than that same light source with a lower dutycycle because it is on for a higher proportion of time. Duty cycle isone aspect of a modulation scheme. Other aspects include pulse shape,frequency of pulses, and an offset level (e.g., a DC bias).

Because DPR modulated light sources 101 rely on frequency modulation,they are able to circumvent the limitations of traditional AM basedapproaches. Note that frequency modulation in this context does notrefer to modifying the frequency of the carrier (which is the lightsignal), but instead to modifying the frequency of a periodic waveformdriving the light source. One popular technique for dimming LED lightsources 101 is pulse-width modulation (PWM), which controls the averagepower delivered to the light source by varying the duty cycle of apulse. In a DPR modulation system utilizing PWM, a DPR modulator wouldcontrol the frequency of the pulses, with the duty cycle determined bythe dimming requirements on the light source 101. As used herein, aDPR-modulated light source, having a DPR modulation frequency, refers toa light source having an output modulated in such a manner that areceiver using DPR demodulation techniques may demodulate the signal toextract data from the signal. In some embodiments, the data may includeinformation in the form of an identifier that distinguishes a lightsource from other nearby DPR-modulated light sources. In someembodiments, this identifier is a periodic tone that the light sourcerandomly selects to identify itself. A periodic tone may be a signalthat repeats with a given frequency. In other embodiments, a lightsource may receive such an identifier from an external source.

To determine the maximum duty cycle (D) supported by DPR demodulation,the modulation frequency (f) of the transmitter and the sampling timefor the image sensor (T_(s)) of the receiver are first defined. Next theduty cycle parameters (T_(off)) and (T_(on)) that correspond to the onand off times of the light source are defined. T_(s) is an importantparameter because the image sensor sampling time defines a minimumamount of modulation time required to produce the banding effects whichallow for the frequency detection required for DPR demodulation. Therequired modulation time may refer to either the T_(on) portion 1804 orthe T_(off) portion 1805 of the signal; however, to maximize thebrightness of the light source, T_(off) is used as the limiting variable(if solving for the minimum duty cycle, T_(on) may be used). If T_(s) ofthe receiving device is less than twice T_(off) of the light source,residual banding on the image sensor will typically not take place;therefore, the signal cannot be extracted. In order for banding tooccur, T_(s) should be greater than twice the value of T_(off)(T_(s)>2×T_(off)).

It is important to note that when designing for the maximum duty cycle,the modulation frequency may be defined from the transmitter side andmay be completely independent of the sampling time T_(s). This isbecause the sampling frequency T_(s) is a property of the receiver,which is defined by the image sensor manufacturer and is likely notdesigned for optimal DPR demodulation properties. T_(s) varies dependingon the specific image sensor, and may be expected to change as moreadvanced image sensors are developed. Therefore, it is important tooptimize such that a broad range of both modulation and samplingfrequencies may be used. In the next sections the equations andvariables for the calculation of the maximum duty cycle are describedfor a variety of test cases.

In order to solve for T_(off) in terms of duty cycle and modulationfrequency, one may first start with the fundamental definition of whatthe duty cycle is: 1 minus the ratio of signal on time divided by thecombination of signal on and off time. In the case of a modulated lightsource, D=1−T_(off)/(T_(on)+T_(off)). Next, the modulation frequency (f)may be defined as the inverse of the sum of signal on and off times:f=1/(T_(on)+T_(off)). Substituting f into the previous equation for Dyields D=1−f×T_(off). The variable T_(off), which was previously definedas a value less than twice T_(s), may then be used to define the maximumduty cycle for any given modulation used in DPR demodulation. Afterrearranging and substituting T_(s) for T_(off) (T_(off)<0.5×T_(s)),D=1−f×(½)×(T_(s)). With this equation, one may now solve for the maximumduty cycle achievable given the modulation frequency of the transmitter,and the sampling time of the receiver.

Since the maximum duty cycle is dependent on both the modulationfrequency of the transmitter and the sampling frequency (F_(s)=1/T_(s))of the receiver, its exact percentage value may change depending on thepresent conditions. For testing purposes, the modulation frequency rangewas chosen to start at 300 Hz, which is above the range which the humaneye can see. The modulation frequency range may range from 60 Hz to 5000Hz. Typical image sensor sampling frequencies (F_(s)=1/T_(s)) rangebetween 20 kHz and 36 kHz for high-quality image settings (640 by 480pixel resolution), and 4 kHz to 7 kHz for low-quality image settings(192 by 144 pixel resolution). In some embodiments, the image sensorsampling frequencies may range from as low as 1 KHz to as high as 1 MHz.

When analyzing specific use cases, the duty cycles corresponding to amodulation frequency of 300 Hz and sampling frequencies for high-qualityimage settings in some embodiments result in D=1−(300 Hz)×(½)×( 1/20Khz)=99.25% and D=1−(300 Hz)×(½)( 1/36 kHz)=99.58%. The duty cyclescorresponding to a modulation frequency of 300 Hz and typical samplingfrequencies low-quality sampling frequencies in other embodiments resultin D=1−(300 Hz)×(½)×(¼ kHz)=96.25% and D=1−(300 Hz)×(½)×( 1/7kHz)=97.86%. In yet other embodiments, a 2000 Hz modulation frequencyand high-quality sampling frequencies of 20 kHz and 36 kHz results inD=95.00% and 97.22% respectively, and for low-quality samplingfrequencies of 4 kHz and 7 kHz results in D=75% and 85.71% respectively.

After the maximum duty cycle has been calculated, to compensate for theadditional power requirements needed for data communication due to theoff portion 1804 of the modulation signal, the input power may beincreased such that the resulting average power of the communicatinglight source 101 is identical to the non-communicating light source 101.In effect, the average power of the two light sources will be the same,yielding a perceivably identical luminous output. Take for instance LEDsource “A” that is powered by 6 watts and modulated where 50% of thetime it is “on”, and the remaining 50% “off”, effectively resulting in a3-watt average power. In order for this light source 101 to match theluminous output of the 6-watt LED source “B” that is not modulating andis on 100% of the time, one may double the input power from 6 watts to12 watts. While the input power of “A” was increased, the average poweris halved to equal 6 watts; therefore, sources “A” and “B” appear to beidentical to the human eye in terms of brightness.

However, there exists a point where increasing the input power maydecrease the efficiency of a given light source 101. For LED lightingdevices it is important to stay within the manufacturer-specifiedvoltage and, more importantly, current, otherwise efficiency drasticallyfalls with increased supply current. This unwanted effect is known asLED “droop,” and generally refers to decreased luminous output for anygiven individual LED (assuming one or more LEDs per lighting source 101)due to the additional thermal heating resulting from the increasedcurrent. In the previous example, the input power to LED source “A” wasdoubled while the input power to “B” was left unchanged. Assuming thateach source was supplied by a constant 12 volts, this means that theinput current to source “A” had to have doubled in order to achieve therequired 12 watts of power consumption. This equates to a 50% increasein current, when moving from 0.5 amperes to 1 ampere, and may only beperformed if within the manufacturers' tolerable input current range forthe LEDs.

Given inputs of drive current (Id) and operating voltage (V), one maydefine the power (P) of a non-modulated light source 101 as P=Id×V, andcompare it with the additional required power (P_(mod)) of a modulatedlight source 101. To define the additional power needed due tomodulation, one may then define the relationship as P_(mod)=P2−(D×Id×V).While the input variables used in this example vary from source tosource, this method may be used to accommodate for power loss due tomodulation.

One may now solve for the power required to support the maximum dutycycles that were previously solved for. In this example, the powerconsumed by the non-modulated light source equals P=Id×V=700 mA×12 V=8.4W. P_(mod) may then be calculated to describe how much extra power isrequired to support a modulated light source 101 with regard to the dutycycle. Recall that for a modulation frequency of 2000 Hz and samplingfrequencies of 20 kHz and 4 kHz, the maximum duty cycle equaled 99.25%and 96.25%. Therefore, the additional power needed to detect a 2000 Hzsignal at a sampling frequency of 20 kHz is defined as Pmod=8.4W−(0.9925×70 mA×12 V)=63 mW, a 0.75% increase in required power on topof the baseline 8.4 W. For 2000 Hz at a sampling rate of 4 kHz,P_(mod)=8.4 W−(0.9625×700 mA×12 V)=315 mW, a 3.75% increase in requiredpower.

While finding the maximum duty cycle supported by DPR demodulation isimportant for maintaining the brightest luminous output levels, it isalso important to support the lowest duty cycle possible in order tosupport the dimmest luminous output levels. This is because the minimumduty cycle corresponds to the dimmest level at which a modulated lightsource 101 may operate at while still supporting DPR demodulation from areceiving device. In order to account for this, one may consider theT_(on) portion of the signal rather than T_(off). The limiting samplingfactor now changes to require that T_(s) is greater than twice T_(on)(T_(s)>2T_(on)). Substituting this condition into the previous max dutycycle equation (replacing {1−D} with D), the resulting equation yieldsD=(½)×f×T_(s).

Repeating the above examples for a modulation frequency of 300 Hz andhigh-quality sampling frequencies (1/T_(s)) of 20 kHz and 36 kHz,D=0.75% and 0.42%, respectively. For a modulation frequency of 2000 Hzwith high-quality sampling frequencies, D=5.00% and 2.78%. Consideringlower-quality sampling frequencies at 300 Hz and 2000 Hz, D=3.75% and2.14% for a 300 Hz modulation frequency, and D=25.00% and 14.29% for a2000 Hz modulation frequency.

In addition to modifying the overall duty cycle, there also exists theopportunity to tune the modulation scheme such that during the “off”portion 1805 of operation the light source 101 does not turn completelyoff. As described in FIGS. 19A-C, modulation schemes 1901, 1902, and1903 depict varying duty cycles where a DC bias 1904 has been addedwhich correspond to the modulated light sources 101 a-101 c. Modulationschemes where the light source 101 does not turn all the way “off” areimportant when considering light source 101 brightness, efficiency,lifetime, and the signal to noise ratio (SNR) of the communicationschannel. The DC bias 1904 during modulation reduces the peak powerrequired to drive the light source for a given brightness. A reductionin peak power will reduce the negative impact of overdriving thelighting source, which is known to cause efficiency losses known as“droop” for LEDs, in addition to decreasing light source 101 lifetimes.

As an example, consider that the average power delivered to the lightsource is defined as: P_(av)=D×P_(on)+(1−D)×P_(off), where D is the dutycycle and P_(on), P_(off) are the respective on/off powers. The impacton light source 101 brightness is that increasing the “off” power willincrease the total power. This reduces the required peak power deliveredto the lighting source, because the power transferred during the “off”period can make up the difference. In a system operating at a duty cycleof 50%, for a fixed brightness B, a 10% increase in the “off” periodpower translates to a 10% decrease in the “on” period power.

When approaching the above power equation from a constant voltage (V),average current (I_(av)), and on/off current (I_(on)/I_(off)) standpoint(P=IV), I_(av)×V=D×I_(on)×V+(1−D)×I_(off)×V. After removing the constantV, I_(av)=D×I_(on)+(1−D)×I_(off). For example, in the case of a lightsource 101 requiring an average drive current (I_(ave)) of 700 mA andoff current of (I_(off)) of OA undergoing modulation with a duty cycle(D) of 96.25%, the peak current (I_(on)) requirement is I_(on)=700mA/0.9625=727 mA. If instead the current delivered during the “off” timeis 100 mA the average current reduces to I_(av)=0.9625×700mA+(1−0.9625)×100 mA=678 mA, a 6.7% decrease in overall required powergiven constant voltage. In other embodiments, a constant current may beapplied with differing voltages to achieve a similar effect.

The impact of non-zero I_(off) values for the previous example istwo-fold. First, a reduction in required power is achieved, and secondincreasing the “off” time power lowers the required duty cycle toachieve a fixed brightness level. For the previous example when solvingfor D, D=(I_(av)−I_(off))/(I_(on)−I_(off)). The difference in duty cyclemay now be determined for the reduction in peak current from 727 mA to678 mA, as D=(700 mA−100 mA)/(727 mA−100 mA)=95.69%, which is a 0.56%difference from 96.25%. This essentially allows for a brighter lightsource 101 with a decreased duty cycle, and lower power requirements.

Another major requirement for DPR modulation is to interface withexisting light dimmers. There are a variety of light source 101 dimmersemployed on the commercial market. One popular dimming technique istriac dimming. In a triac dimmer, a variable resistor switch is used tocontrol the amount of power delivered to the light source 101 over theAC line. For traditional incandescent and fluorescent sources this is acost-effective and efficient way to control the power, and thus thebrightness, delivered to the light source 101. For LED light sources101, it is necessary to put a special driver between the triac dimmingcircuit and the LED source. This is because LEDs are current-drivendevices, and thus require an AC/DC converter to transform AC from thepower lines to a DC current for driving the LEDs.

FIG. 20 demonstrates a system by which a DPR modulator may interfacewith existing lighting control circuits. A dimmer controller 2002 sendsa dimmer signal 2003 to a dimmable LED driver 2006. In the case of anLED light source controlled by a triac dimmer, the dimmer signal wouldbe transmitted across the AC power line. The dimmable LED driver 2006then converts the dimmer signal to a pulse width modulated signal usedfor driving the light output 2007 of the source 2001. The configurationof the system diagram shows the dimmer signal 2003 going to both the DPRmodulator 2004 and the LED driver 2006; however, this does not alwaysneed to happen. In some instances the LED driver 2006 may contain a“master override” input that is designed to supersede any dimmer signal2003 input. In this case, the dimmer signal 2003 still goes to the LEDdriver 2006, but is ignored. In other cases where there is not anoverride input, the dimming signal only goes to the DPR modulator.

DPR modulator 2004 is responsible for sending DPR signals 2005 to theLED driver 2006 that controls the light output 2007. In the case of thelight source 2001 being driven by pulse-width modulation as the dimmersignal 2003 from the dimmer controller 2002, DPR modulator 2004 controlsthe frequency of the PWM signal and selects the desired value. The widthof pulses in signals 1801-1803 are determined based on dimmer signal2003, which indicates the desired light source 2001 brightness level.Note that the dimmer controller 2002 is not contained within the lightsource 2001, and may output a variety of dimmer signals 2003 (triac, ora proprietary method). Because of this, the DPR modulator 2004 isresponsible for interpreting these different signals and appropriatelyoutputting a DPR signal 2005 that corresponds to the desired brightnesslevel of the inputted dimmer signal 2003. In cases where dimming is notrequired and the dimmer signal 2003 is not present, the DPR modulator2004 interfaces directly with the LED driver. In some implementations,the DPR modulator 2004 may also be contained inside the LED driver 2006as part of an integrated solution instead of as a separate component.

FIG. 21 contains a high level overview of a DPR modulator 2004. Data2101 is first sent to DPR tone generator 2102. Data 2101 may containinformation from any source. In the context of a beacon-basedlight-positioning system, data may include the identifier for the light.DPR tone generator 2102 converts the data 2101 into a sequence of DPRtones. A DPR tone is a periodic digital signal that oscillates betweenactive and inactive states with a particular frequency. This process isdescribed further in FIG. 22. Depending on the requirements of the datatransmission channel, this could either be a single tone (suitable for abeacon based positioning system using light identifiers), or a sequenceof tones (if higher data rates are desired by the end user). The DPRTone(s) 2203 are then sent to the waveform generator 2103, which isresponsible for generating the DPR signal 2005 for driving the LEDs.Waveform generator 2103 receives a dimmer signal 2003 input from adimmer controller 2002, which controls the brightness of the lightsource. In the case of a DPR tone as a pulse-width-modulated signal,dimmer controller 2002 would control the duty cycle of square wave 1802,while DPR Tone(s) 2203 would control the frequency of the square wave.The result is an output DPR signal 2005, which is then sent to the LEDdriver 2006.

FIG. 22 contains a breakdown of DPR Tone Generator 2102. This module isresponsible for taking a piece of data and converting it to a sequenceof DPR tones. A DPR tone determines the frequency at which a waveform,such as the square waves from FIG. 18, is sent. The range of possibletones, defined here in as T_(o) through T_(n), is determined by both thesampling time, T_(s), of the image sensor (as discussed in paragraph0006), and the frequency response of the light source 101. Encoder 2201is a standard base converter—it takes a piece of data in binary andconverts it into a corresponding DPR tone. A typical range for tonescreated by DPR Tone Generator 2102 is 300 Hz-2000 Hz, in steps of 10 Hz,allowing for 170 distinct DPR tones. The step size between tones isselected to reduce noise, and depending on the requirements could bemuch higher or lower than 10 Hz. As an example, that data 2101 maycontain an identifier of value 10 for light source 101. This identifieris passed to Tone(s) Generator 2102, which generates (or selects frommemory) a sequence of tones. Note that the length of a DPR tone sequencecould be as low as 1 (in the case of a single tone used in abeacon-based positioning system). In this example, an identifier of 10would map to a DPR tone of 400 Hz. DPR Tone Generator 2102 could eitherstore the identifier in memory beforehand, using pre-computed mappingsof data to tone sequences, or alternatively it could compute this on thefly. The exact method of generating the sequence of tones may be drivenby the resources available on the light source 101. Once one of thepossible tones sequences 2202 is created, it is sent to WaveformGenerator 2103.

FIG. 23 contains the breakdown of Waveform Generator system 2103, whichcombines a tone sequence 2202 with a waveform from symbol creator 2303and dimmer signal 2003 to create a DPR signal 2005 for driving lightsource 101. The resulting waveform will be periodic, with a frequencydefined by the sequence of tones, a symbol created based on the list ofpossible symbols in symbol creator 2303, and an average output(brightness) determined by the dimmer signal 2003. This desiredbrightness could either be hard-coded on the module, or provided as anexternal input through a dimming control module. The choice of a symbolis determined within Symbol Selector 2301, which generates a controlline 2302 for selecting a symbol from symbol mux 2402.

FIG. 24 contains the breakdown of Symbol Creator 2303, which holdspossible symbols 2401 a-2401 d. These could include a saw tooth wave2401 a, sine wave 2401 b, square wave 2401 c, and square wave with a DCoffset 2401 d, or any other periodic symbol. Symbol creator then takesin a selected symbol 2402, and modifies it such that a desiredbrightness 2106 is achieved. In the case of a square wave symbol 2401 c,dimmer signal 2003 would modify the duty cycle of the square wave. Theresulting waveform is then sent to output signal 2005 for driving thelight source.

The goal of the output waveform 2105, which drives light source 101, isto illuminate a scene in such a way that the DPR modulated signal may bepicked up on any standard mobile device 103. Reducing flicker on videowhich is under illumination from fluorescent lamps is a well-knownproblem. The flicker is caused by periodic voltage fluctuations on theAC line powering the lamp. For a lamp powered by a 50 Hz AC line, theluminance level changes at 100 Hz. This causes alternating white/darkbands to appear in video recorded with CMOS imagers. The bands are aresult of the rolling shutter mechanism on CMOS imagers, which partiallyexpose different areas of the image at different points in time. Thelines on the image may occur on both, one, or on multiple frames, andmay appear to move in time. See, for example, U.S. Pat. No. 6,710,818,the entire contents of which is hereby incorporated in its entirety,which describes methods for detecting and removing this unwanted effect.Possible algorithms for mitigating flicker include automatic exposurecontrol, automatic gain control, and anti-banding. These techniques arecommon in many mobile devices as a means to remove flicker caused byfluorescent lamps.

Advanced DPR Demodulation Techniques

DPR demodulation, instead of removing flicker, exploits the rollingshutter effects of CMOS cameras as a means of transmitting data. A CMOSdevice with a rolling shutter captures an image frame by sequentiallycapturing portions of the frame on a rolling, or time-separated, basis.These portions may be vertical or horizontal lines or “stripes” of theimage that are captured at successive time intervals. Because not everystripe is captured in the same time interval, the light sourcesilluminating the image may be in different states at each of these timeintervals. Accordingly, a light source may produce stripes in a capturedframe if it is illuminated in some time intervals and not illuminated inother time intervals. Light sources that broadcast digital pulserecognition signals may produce patterns of stripes. Since the patternof stripes is dependent on the frequency of the digital pulserecognition signal, and the speed of the rolling shutter can bedetermined a-priori, image processing techniques may be used to deducethe illumination frequency based on the width of the stripes. Forexample, consider a room containing five light sources 101, eachbroadcasting at 500 Hz, 600 Hz, 700 Hz, 800 Hz, and 900 Hz,respectively. Each distinct frequency, otherwise known as a DPR tone,may be used to identify the light source 101. In a beacon-basedlight-positioning system, a mobile device receiver within view of thetransmitting lights can detect the DPR tones, correlate an identifierassociated with the tone, and then use a lookup table to determine thelocation of the device based on the location associated with theidentifier(s).

Modeling the camera sampling function is advantageous in understandinghow DPR demodulation works on modern image sensors, and how the impactsof various hardware-dependent parameters affect the DPR signal 2105. Torepresent this, FIG. 25 is a continuous time representation 2501 of howan individual row on a rolling shutter image sensor is sampled. Theexposure time interval 2502 represents the period over which lightaccumulates on the photo sensor. If the exposure time is much lower thanthe period of the DPR modulated signal, the light and dark bands will beclearly defined. If the exposure time is longer, the light and darkbands will lose their definition.

FIG. 26 contains a continuous time example 2601 of a DPR modulated lightsignal. In this example, the signal is a square wave with a 50% dutycycle being driven at a DPR tone of 300 Hz. The relationship between theDPR illumination period 2602 and the exposure time 2502 determines howwell defined the bands are on the received image.

FIG. 27 is the continuous time sampled image 2701, created by convolvingan individual row sampling function 2501 with a DPR modulated signal2601. The alternating periods of high brightness 2702 and low brightness2803 are caused by the DPR modulation frequency, and appear asalternating white/dark bands on the received image.

FIG. 28 is a representation of a discrete time-domain signal model 2801for representing how a rolling shutter on an image sensor samples theincoming light pulses 2601. The rolling shutter is modeled as an impulsetrain, containing a sequence of the Dirac Delta functions (otherwiseknown as a Dirac comb). Each impulse is separated by an interval, T,which corresponds to the speed of the rolling shutter commonly found inmost CMOS image sensors. The interval T varies from device to devicewhich causes the bands on scenes illuminated by DPR modulated signals tovary in size. The mobile device 103 preferably accounts forhardware-dependent factors (e.g., rolling shutter speed) to properlydetermine the DPR tone. FIG. 29 contains a discrete time representation2901 of the rolling shutter sampling functionality over multiple frames.

Because rolling shutter speeds are typically faster than frame rates,DPR demodulation on current imaging technology is capable of much higherdata rates than modulation schemes that sample on a per-frame basis. Ina DPR modulated system using a 640×480 pixel image sensor, the sensorwould capture 480 samples per frame (represented as 480 consecutivedelta functions in sensor model 2801). A demodulation scheme using aglobal shutter would only be capable of taking one sample per frame.This is a key advantage for indoor positioning using beacon-basedbroadcasting schemes because the time-to-first-fix is orders ofmagnitude faster than competing technology, which may take severalseconds to receive a signal. For example, consider a typical mobiledevice 103 camera which samples at 30 frames per second (FPS). Using DPRdemodulation, time-to-first-fix may be achieved with as little as asingle frame, or 1/30 of a second, versus 1 second for a demodulationscheme that samples on a per-frame basis. This compares to atime-to-first-fix of up to 65 seconds for GPS, 30 seconds for assistedGPS, and 5-10 seconds for WiFi positioning.

This order of magnitude improvement opens the door for applications inwhich latency for time-to-first-fix must be minimized. Furthermore,computation for DPR demodulation may be performed on the mobile deviceitself, versus the server-side processing required for WiFifingerprinting algorithms. In a mobile environment, where connection toa network is not guaranteed, client-side processing provides a majoradvantage. In the future, it is expected that image sensors will havemuch higher frame rates. In this scenario, DPR demodulation may beadjusted to sample on a per-frame basis, instead of a rolling shutterbasis. The key principle is that the demodulator may be adjusted insoftware, allowing future mobile devices to tune their receivingcharacteristics to receive DPR signals. The software adjustments thatneed to be applied are the subject of the following sections.

Configuring a Device for DPR Demodulation

In order to prepare a mobile device 103 to receive the modulated DPRsignals 2105, the device is first configured. This is to counteract theflicker-mitigation algorithms typically applied in mobile device imagesensors. FIG. 30 describes the method by which mobile device 103 isconfigured to receive DPR modulated signals. First, the initializesensors 3001 function initializes and activates the available sensorscapable of receiving data. For typical modern mobile devices these wouldinclude both the front- and rear-facing cameras. Here, a “front-facing”camera or other sensor of a mobile device is one that is mounted on thesame side of the device as its display and is therefore likely to facetoward a user. In one preferred embodiment, the rear-facing camera oranother rear-facing is used because it is more likely to have a view ofthe user's surroundings that is relatively unoccluded by the user's ownbody and thus to record light cast directly by local light sources.Determine sensors to modify 3002 then decides which sensors need to bemodified. A number of possible factors determine whether or not aparticular sensor should be initialized then modified, including powerconsumption, accuracy, time since last reading, environmentalconditions, required location accuracy, and battery state.

Modify sensors 3003 then passes a list of the appropriate sensors whichneed to be modified to a function which has additional information aboutthe mobile device 103 and adjusts the demodulation scheme for devicespecific limitations 3004. In the case of using an embedded mobiledevice 103 camera to demodulate DPR signals, possible sensor parametersto modify include exposure, focus, saturation, white balance, zoom,contrast, brightness, gain, sharpness, ISO, resolution, image quality,scene selection, and metering mode. As part of the modification step3003, sensor parameters such as exposure, white-balance, and focus arelocked to prevent further adjustments.

After the sensors are modified 3003, specific hardware limitations areadjusted for in the demodulation scheme by using a device profile. Themost important of these is the rolling shutter speed. Because differentmodels of mobile device 103 will, in general, have different camerasensors, the line width of the DPR tone measure on an image sensor willvary across hardware platforms for a fixed frequency. For this reason,it is necessary to adjust the stripe width one is looking for dependingon the specific characteristics of the device. In the Fourier Techniquesdiscussed later on in the application, modifying the stripe widthcorresponds to modifying the sampling frequency of Dirac Comb 2801.

There are a number of challenges associated with controlling the cameraparameters to optimize for DPR demodulation. One challenge is overridingthe automatic parameter adjustments that mobile operating systemstypically provide as part of their camera application programminginterfaces (APIs). In the case of an embedded image sensor, the sensorsettings are adjusted automatically depending on factors such as but notlimited to ambient light conditions, areas of focus, distance fromobjects, and predetermined scene selection modes. For instance, whentaking a picture with an image sensor, if the scene is dark then theexposure time is automatically increased. When taking picture of a scenemode with fast moving objects, the exposure time is usually decreased.

When using an image sensor for DPR demodulation, these automaticadjustments may introduce noise into the signal, causing higher errorrates. Specifically in the case of exposure, longer exposure timescorrespond to lower data rates, which correspond to a decreased amountof available light IDs 901. At the edge case, if the exposure time issufficiently long, then the sampling rate will drop so low that DPRdemodulation becomes extremely challenging as the signal is severelyunder-sampled. Furthermore, if the camera is constantly adjusting, thenthe performance of background subtraction (discussed later), whichisolates the moving stripes from the rest of the picture, will besignificantly impaired. This is because the automatic adjustments areconstantly changing the pixel values. In order to successfully transmitDPR signals, these automatic adjustments need to be accounted for.

Practically speaking, many mobile device 103 APIs do not allow for themodification of sensor parameters in the top-level software. Theproposed method in FIG. 31 describes a method for working around theprovided APIs to control the exposure. Current APIs do not allow formanual exposure control, so instead of manually setting the exposure, analgorithm is presented that exploits the metering functionality tominimize the exposure time.

FIG. 31 contains a process for modifying the various sensor parameterscontained in a mobile device 103 in a way that overcomes the limitationsimposed by current camera APIs. In the algorithm, the first step is toinitialize the required sensors 3001. For the case of an image sensor,this involves setting the frame rate, data format, encoding scheme, andcolor space for the required sensors. After the image sensors have beeninitialized 3001, the algorithm searches for regions of interest 3101.In the case of setting the exposure using metering, these regions ofinterest 3101 would be the brightest regions of the image. Set meteringarea 3102 then sets the metering area to the brightest portion,effectively “tricking” the mobile device 103 into lowering the exposuretime. Lock parameter 3103 then locks this exposure time to prevent theauto-adjustment feature of the camera from overriding the manualsetting. Next, adjust for hardware dependent parameters 3104 accesses alookup table and adjusts the demodulation algorithm based on hardwareand software differences. For the case of an image sensor, one exampleof this is changing the sampling time based on the rolling shutter speedof the device. This rolling shutter speed may either be loaded from alookup table beforehand (using predetermined values) or measured on thefly. Each device only needs to measure its rolling shutter speed onceper image sensor. Once parameters set? 3105 is satisfied the algorithmends; otherwise, it returns to identify regions of interest 3101.

The method of exploiting the metering area on a mobile device 103 may beused to optimize many of the required parameters in addition to theexposure, including white balance, contrast, saturation, ISO, gain,zoom, contrast, brightness, sharpness, resolution, image quality, andscene selection. Furthermore, these parameters could already be knownbeforehand, as each mobile device 103 will have its own “device profile”containing the optimal camera settings. This profile could be loadedclient side on the device, or sent over a server. Note that although themethod of using the metering area to control the exposure may improvethe performance of DPR demodulation, it is not strictly necessary.Simply locking the exposure 3103 is often sufficient to prevent theautomatic camera adjustments from filtering out the DPR signals.

Advanced Techniques for Decoding Information in DPR Modulated Signals

Once the sensors have been initialized 3001 and parameters have been set3104, FIG. 32 describes a process for decoding the information containedinside a DPR modulated signal. Identify regions 3201 is used to separatedifferent regions on the image illuminated by DPR signals. At the baselevel, the region of interest is the entire image. However, when one ormore light sources 101 are present, there exists an opportunity toreceive multiple DPR signals simultaneously. In this scenario, thesensor effectively acts as a multiple antenna receiver. Such multipleantenna systems, more generally referred to as multiple-inputmultiple-output (MIMO), are widely used in the wireless networkingspace. This is an example of spatial multiplexing, where wirelesschannels are allocated in space as opposed to time or frequency. Theimplications of MIMO for DPR demodulation in a beacon-basedlight-positioning system is that frequencies may be re-used in a spacewithout worry of interference. When a mobile phone user receives DPRmodulated signals on a photodiode array (such as an image sensor, or anyimaging technology that contains multiple spatially separated sensors),the DPR signals will each appear at different locations on the sensor.Each region 3201 of the image may then be processed independently, inthe same way that each mobile phone user in a cell network only connectsto the cell they are closest to.

This works in a way analogous to cellular phone networks. With cellularnetworks, mobile phone users only communicate with cellular towers thatare close to them. This allows multiple mobile phone users to share thesame frequency, provided they are all on different cells. In DPRmodulation, each light acts as its own cell transmitting uniquefrequencies. However, different lights may also use the same frequencyprovided that they are far enough apart. Re-using the same frequenciesin different space allows for greater system scalability, since lightingsources 101 may be installed at random without requiring the installerto worry about frequency allocation.

After sensors have been initialized 3001, and regions of interest 3201have been identified, detect frequency content 3202 identifies thepresence of DPR tones from the sensor data. Described here are multiplemethods for extracting the frequency content from a DPR signal. Onepossibility is to use line-detection algorithms to identify the pixelwidth of the stripes, which directly corresponds to the transmittedfrequency. This stripe width is then used to access a lookup table thatassociates width and transmitted frequency and determines thetransmitted tones. Possible methods for detecting lines include Cannyedge detection, Hough Transforms, Sobel operators, differentials,Prewitt operators, and Roberts Cross detectors, all of which are welldeveloped algorithms, known to those of skill in the art. Adjust fordependent parameters 3004 then modifies the appropriate camera sensorsfor optimal DPR demodulation. In the case of line detection, thiscorresponds to a linear adjustment for the line width lookup table.Determine tones 3203 uses the adjusted line width to determine the DPRtone sent. This process is performed for each region on the image, untilthere are no more regions 3204 remaining. A data structure containingall the regions, with their associated identifiers, is then returned3205.

An additional method for performing DPR demodulation is described inFIG. 33. One or more light sources 101 illuminates a scene 3301. Whenthe image sensor on mobile device 103 acquires a sequence of images3302, the brightness of any given pixel depends on both the details ofthe scene as well as the illumination. In this context, “scene” refersto the area within view of the camera. The scene dependence means thatpixels in the same row of the image will not all have the samebrightness, and the relative brightness of different image rows is notsolely dependent on the modulated illumination 3301. If one were to takethe Fourier transform of such an image, both the frequency content ofthe illumination, as well as the frequency content of the underlyingscene, will be present.

In order to recover the frequency content of the modulated illuminationindependently of the scene, the contribution of the scene may be removedusing a background subtraction algorithm 3303. The “background” is theimage that would result from un-modulated illumination as opposed to theeffects of modulated illumination 3301. Subtracting the background froman image leaves only the effects of illumination modulation. Onepossible implementation of a background subtraction method uses a videosequence. If a video of a scene illuminated with modulated light isrecorded, the light and dark bands may appear at different locations ineach frame. For any modulation frequency that is not an exact multipleof the video frame rate, there will be a resulting beat frequencybetween the video frame frequency and the illumination modulationfrequency. The illumination signal will be in a different part of itsperiod at the beginning of each frame, and the light and dark bands willappear to be shifted between video frames (i.e. the bands will appear tomove up or down across the scene while the video is played). Althoughthis algorithm is described with the use of a video sequence, otherembodiments may perform background subtraction using still images.

Because the bands move between video frames, the average effect of thebands on any individual pixel value will be the same (assuming that in along enough video each pixel is equally likely to be in a light or darkband in any given frame). If all the video frames are averaged, theeffects of the bands (due to the illumination modulation) will bereduced to a constant value applied to each pixel location. If the videois of a motionless scene, this means that averaging the video frameswill remove the effect of the bands and reveal only the underlying scene(plus a constant value due to the averaged bands). This underlying scene(the background) may be subtracted from each frame of the video toremove the effects of the scene and leave only the effects ofillumination modulation 3301.

FIG. 34 contains an implementation of a possible background subtractionalgorithm 3304. A frame buffer 3402 accumulates video frames 3401. Thesize of this buffer can vary, depending on the memory capacity of mobiledevice 103 and the required time to first fix. Frame averaging 3403computes the average based on the frames in the buffer 3402. The averageof these frames is used to generate background frame 2704. Thebackground frame may be acquired using a number of different averagingtechniques 3403, including a simple numerical average, a normalizedaverage (where each frame is divided by the sum of all the frames),Gaussian averaging, or by doing a frame difference between subsequentframes. A frame difference simply subtracts subsequent frames from oneanother on a pixel-by-pixel basis.

For video of a scene with motion, simple averaging of video frames willnot yield the underlying scene background. FIG. 35 describes a techniquefor dealing with motion between frames, which is a likely scenario whendemodulating DPR signals on mobile device 103. Motion compensation 3501is necessary to best determine the underlying scene. By determining themotion between video frames (for example, shifting or rotation of thewhole scene due to camera movement), each video frame may be shifted ortransformed such that it overlies the previous frame as much aspossible. After performing these compensatory transforms on each framein motion compensation 3501, the video frames are averaged 3403 to getthe scene background 3404. Phase correlation is one possible method ofestimating global (i.e., the whole scene moves in the same way, as inthe case of camera motion while recording video) translational motionbetween frames. The 2D Fourier transform of a shifted image will be thesame as that of the original image, except that a phase shift will beintroduced at each point. Normalizing the magnitude of the 2D Fouriertransform and taking the inverse transform yields a 2D image with a peakoffset from the center of the image. The offset of this peak is the sameas the shift of the shifted image. Those skilled in the art willrecognize that additional methods for motion compensation 3501 includeKernel Density Estimators, Mean-shift based estimation, andEigenbackgrounds.

After removing the background scene, Fourier Analysis may be used torecover the DPR tone based on signals received from modulated lightsource 103. Specifics of this method are further described in FIG.36-43. FIG. 36 contains a sample image 3601 of a surface illuminated bya light source undergoing DPR modulation. The image is being recordedfrom a mobile device using a rolling shutter CMOS camera. The stripes3602 on the image are caused by the rolling shutter sampling function,which is modeled in by the sequence of Dirac Combs 2801 in FIG. 28.

FIG. 37 shows the result 3701 of performing background subtraction onthe raw image data from FIG. 36. Background subtraction is used toextract the stripes from the raw image data. The result is an image ofalternating black/white stripes that represents the discrete time-domainrepresentation of the transmitted DPR signal. The stripes 3702 are muchmore pronounced than in the raw image data from FIG. 36 due to theimprovement from background subtraction.

Illumination modulation affects each row of a video frame identically,but imperfect background subtraction may lead to non-identical pixelvalues across image rows. Taking the Fourier transform of row valuesalong different image columns, then, may produce different illuminationsignal frequency content results. Because the true illumination signalfrequency content is the same for the entire image, a technique toreconcile these different results may be employed. One possible methodis to assign the average pixel value for any given row to each pixel inthat row. This method takes into account the information from each pixelin the row, but by yielding uniform row values gives a singleillumination signal frequency content result when taking the Fouriertransform of row values along an image column. FIG. 38 displays theresults of applying row averaging 3801 to the background subtractedimage 3701. The stripes 3802 are much more visible as a result of therow averaging, and they are also more consistent across rows.

FIG. 39 shows the Fourier transform 3901 of the row averaged image 3801from FIG. 38. There is a peak frequency at the DPR tone of 700 Hz, aswell as a DC component at 0 Hz. The peak frequency is used to identifythe sequence of tones, and thus the transmitted identifier.

FIG. 40 shows the Fourier transform 4001 from FIG. 39 after applying ahigh-pass filter. The DC component of the signal is removed, whichallows a peak frequency detector to move to detection of the DPR tonefrequency.

FIG. 41 shows a 2-D Fast Fourier Transform 4101 of the post-processedDPR modulated signal data 3701. In comparison to the 1-D Fourieranalysis performed in FIGS. 38-40, 2-D Fourier analysis of the DPRmodulated signal 3601 may also be performed. 2-D Fourier Analysis is apopular and widely used technique for image analysis. Because there area number of software libraries that are highly optimized for performingmultidimensional FFTs, including OpenCV, multidimensional Fourieranalysis is a viable alternative to the 1-D analysis. The DPR tones 4102may be easily seen across the vertical axis 4103 of the 2-D FFT.Brighter areas on the FFT image 4101 correspond to areas on the imagewith higher spectral content. A peak may be seen at the origin 4104,which corresponds to the DC component of the DPR signal.

FIG. 42 shows a low-pass filtered version 4201 of the 2-D FFT 4101. Thefiltered image 4201 contains dark areas 3502 at the higher frequencieson the image. The low pass filter rejects the higher frequencies. Thisis a key component of successful DPR demodulation. As discussedpreviously, DPR modulation relies on transmitting digital signals atdifferent frequencies. When using Fourier analysis on these signals,higher frequency harmonics appear, in particular at higher duty cycles.These higher frequency components act as noise in the signal, soremoving them with filtered image 4201 is one technique for recoveringthe transmitted tones.

When performing spectral analysis in the case of a 1-D FFT 3901 in FIG.39, it was necessary to remove the DC component of the DPR signal. PWMsignals 1901-1903 will contain a significant DC component, which needsto be filtered before moving on to extract the transmitted DPR tone.FIG. 43 shows a high-pass filtered version 4301 of the 2-D FFT 4101. Thedark area 4302 at DC demonstrates the result of the high-pass filter,which rejects the DC noise component. The higher frequency bands 4303are still contained in the signal, allowing the demodulator to determinethe peak frequency.

An important consideration in the design of a beacon based lightpositioning system is the choice of light identification codes withrespect to the system layout. In a system which uses individual digitalpulse recognition (DPR) tones to identify the positions of lightsources, it is desirable to lay out the light sources such that adjacentsources have frequencies that are spaced far apart. For example,consider FIG. 1, which contains light sources 101 a-d, each with its ownDPR tone. When a mobile device user 102 moves underneath beacon basedlight sources 101 a-b, each of which is emitting a DPR tone, there isthe possibility that the user's mobile device 103 will receive multiplelight signals simultaneously. In such a situation, if the spacingbetween the tones is too low (for example, in the case of light source101 a emitting a DPR tone of 700 Hz, and 101 b emitting a tone of 705Hz), then spectral leakage may occur. Spectral leakage is a well-knownphenomenon associated with Fourier analysis. It is a consequence of thefinite observation time over which a signal is measured. For DPRdemodulation that exploits the rolling shutter, this finite timecorresponds to the period of the rolling shutter. Spectral leakagenegatively impacts the ability to distinguish two or more frequencies,so in the case of user 102 receiving multiple DPR tones, the ability todistinguish tones will be diminished. In some cases, the tones will beunresolvable, which could possibly cause dead spots in the beacon basedpositioning system presented in FIG. 2.

The impact of spectral leakage can be mitigated in a number of ways. Onetechnique is the use of a digital filter, which is sometimes referred toas a window function. Popular choices for window functions includeRectangular, Hann, Hamming, Turkey, Cosine, Kaiser, and Gaussianwindows. Using a window function allows one to improve the spectralresolution of the frequency-domain result when performing Fourieranalysis.

A simple approach to reduce the impact of spectral leakage is to controlthe spatial distribution of bulbs. For example, one could come up with arule which says that no two adjacent bulbs can have a DPR tone within 50Hz of another adjacent bulb. For most lighting topologies 4403, mobiledevice users 4402 a-b will see at most two adjacent lights at the sametime. If the DPR tones are distributed such that adjacent bulbs aredissimilar enough, then the individual DPR tones can be resolved.

Furthermore, in cases in which the DPR signal is more complex, andpossibly composed of several tones transmitted simultaneously, adjacentlight sources could destructively interfere with each other. Forexample, consider that the two light sources 4401 a and 4401 b in FIG.44 each transmit multiple DPR tones. Light source 4401 a emits 300 Hz,400 Hz, and 500 Hz, while light source 4401 b emits 500 Hz, 600 Hz, and700 Hz. Note that both sources share a common tone of 500 Hz. Since thebulbs are typically non-networked, their emissions are not synchronized.Accordingly, if the common 500 Hz tones are out of phase with oneanother, they will destructively interfere, possibly resulting in amissed detection.

FIG. 47 depicts a user commissioning a beacon based light positioningsystem. Commissioning refers to the process of acquiring of acquiringDPR signals being emitted from light sources 4401 a-4401 b, and thengeoreferencing the light sources. Mobile device user 4702 a invokes anapplication running on their mobile device 4703. The application listensfor the DPR signals using sensors on the device. When the mobile devicedetects a valid DPR signal, the user is prompted to georeference thelight source. This could be done by simply entering in coordinates(which could take the form of latitude, longitude, and altitude), orsome arbitrary coordinate system. The user could also assign thecoordinates by dragging and dropping the light location on a map. Oncethe coordinates are assigned, they can be sent to a remote database 802for storage. The database contains records which associate the lightidentifier with the light location. A user interface developed for thistask is presented in FIG. 48.

After a user georeferersces light source 4401 a, they proceed in a path4704 a underneath additional light sources 4401 a-c. For each lightsource, as they successfully detect the source, they are prompted togeoreference the light. If the user attempts to add a light source thatviolates the rules for the lighting topology (as in the previousexample, where adjacent lights are not allowed to contain DPR tones lessthan 50 Hz apart), the user would be prompted to change the location ofthe light source before adding it to the map.

FIG. 47 also contains a situation by which multiple users 4701 a-b cancommission a space simultaneously. Note that the number of users couldbe much larger than two, especially in the case of a large space. Havingmultiple users operate at the same time greatly speeds up thecommissioning process. As users 4701 a and 4701 b move about theirrespective paths 4704 a and 4704 b, they georeference the lightlocations in the same manner that an individual user would. Each timethe user adds a light, the light could be sent to a remote server 703for storage. The connection to the server could be done through anystandard network connection. Mobile devices 4702 a-b would each writetheir respective portions of the georeferenced light positions to theserver.

The user workflow for multiple users working in conjunction tocommission a space could come in a variety of forms. In one embodiment,each mobile device user 4702 a-4702 b would maintain a synchronousrepresentation of the current light georeferencing. In this scenario,when mobile device user 4702 a adds a light to the database, mobiledevice user 4702 b would see that light location appear on their device.This helps to avoid the need for the users to maintain closecommunication during commissioning, thereby speeding the process along.In another embodiment, mobile device users 4702 a-b would each maintaintheir own separate records of the light locations. When the remoteserver 703 received the commissioning data from, each user, it wouldthen take all of the data as input and determine the best result. Forexample, in one embodiment the server would take a simple numericalaverage of the coordinates for each light. By combining data from manyusers when commissioning a space, the accuracy of the system isimproved.

Note that in this embodiment the users are depicted as the onescommissioning the space. However, in another embodiment thiscommissioning could be performed automatically by a non-human entity, orrobotic agent that patrols the space. The robot can use forms ofinertial navigation techniques and sensor fusion to maintain an estimateof its current position. The robot must first be assigned an initiallocation fix, which could be done through a manual input, or a locationfix acquired from an alternative positioning technique such as WiFi,Bluetooth, GPS, A-GPS, Ultrasound, Infrared, NFC, RF-ID, a priori knownlocation, or markers in the visible or non-visible spectrum. When therobot receives a light identifier from light source, it then uploads theidentifier along with its estimated position to the light locationdatabase. Furthermore, the robot can also send an estimate of thestrength of the current signal, along with an error estimate of itscurrent position, as well as any additional relevant information. Theserver can then combine all of this information to get a more accurateestimate of the light position. This is analogous to the way the servercombined information from multiple users in FIG. 47.

FIG. 48 contains one possible user interface for commissioning a spacein a light positioning system. As depicted in FIG. 47, mobile deviceusers 4702 a-b commission a space using mobile devices 4703 a-b. Themobile devices 4703 a-b presents location information, such as a map4806, of the space. When a user 4702 a acquires a signal emitted from alight source 4401 a, a user notification, such as an add button 4801,becomes active on the user interface. The user can then add the lightonto the map by “dragging and dropping,” using either a point and clickgesture with a mouse, keyboard based commands, a touchscreen, or othersuch user interfaces that are available. After a light has been placedon the map 4806, an icon 4804, or identifier, appears to mark itslocation. Additional user feedback, such as making the device vibrate orchange color, can be used to make the experience more tactile. The usercan use the move button 4803 to move the location of the light in thespace. In addition to manual location edits, additional information suchas an architectural, electrical, or mechanical building plan can be usedto assist in light placement or to automate fine tuning of the process.Furthermore, the user can edit and delete lights from the map using theEdit button 4802. Once a light is placed on the space, and the marker4804 has been assigned a location, the marker may change its visual look4805 when the user walks underneath the appropriate light andsuccessfully receives the DPR signal. The visual feedback could includechanging the color, shape, or moving the marker around on the screen.This makes it easier for the user to verify the position andidentification code of the light after they have commissioned the space.

One aspect of DPR demodulation in a light positioning system includesadjusting receiver parameters depending on specific characteristics ofmobile device 4901. As discussed previously in the applications citedabove, parameters such as rolling shutter speed, frame rate, andexposure time can all vary across different hardware platforms. In orderto compensate for these differences, adjustments can be created insoftware. One method of dealing with this involves creating andmaintaining a database of device profiles, as depicted in FIG. 49. Adevice profile can include all the relevant information regarding thereceiver parameters of a mobile device 4901. For example, this deviceprofile could include the exposure time, sampling frequency, number ofimage sensors, or other hardware dependent parameters for a specificversion of mobile device 4901. When the mobile device performs DPRdemodulation as part of a light positioning system, the device profileis first loaded. The profile could either be fetched from a remoteserver, or stored locally on the device. The parameters contained withinthe device profile are then fed as inputs into the demodulationalgorithm.

Device profiles can either be created in a number of different ways. Inone embodiment, the device profile could be created via manual input ofvalues. A system administrator could acquire device information usingsensor datasheets, or other sources, and then manually input theparameters into the database. In another embodiment, the mobile devicecan utilize a self-calibration algorithm to figure out its own deviceprofile. For example, consider a mobile device, with an unknown deviceprofile that is positioned within view of a DPR modulated light sourceemitting a known tone. The mobile device could enter as an input thetone which it is supposed to be seeing, and then adjust its own deviceparameters such that the output of the detection algorithm is equal tothe tone that is “known.”

Self-calibration is another technique for resolving differences betweendevice hardware and software. In this method, a special calibrationfrequency can be transmitted at the beginning of a DPR packet, allowingthe mobile device 103 to measure its own sensor parameters, such as theshutter speed, and adjust accordingly. The downside to this approach isincreased read time on the receiver, because the calibration framecarries no positioning information.

Alternative Algorithms for DPR Demodulation

In addition to the DPR demodulation algorithms presented earlier, thereare a number of alternative techniques for demodulating DPR signals.These algorithms are presented in the sections that follow, and includeboth Fourier and non-Fourier based methods. It should be understood thatthese algorithms illustrate exemplary DPR demodulation algorithms, andsimilar equations are also within the scope of this disclosure.

When captured by a rolling shutter image sensor, a DPR modulated signalproduces periodic horizontal bands that overlay the entire 2D content ofa given scene. An M×N frame (M columns, N rows) can be viewed as a setof M 1D signals y_(o, . . . , m-1) of length N. Since each column issubject to the same modulation signal, the phase of frequency componentsY_(o, . . . , M-1)[k_(BL)], where k_(BL) is the vertical spatialfrequency of the periodic horizontal bands, is expected to be identicalacross all columns y_(o, . . . , m-1), i.e. Var(∠Y_(o, . . . , M-1)[k_(BL)])=0. In practice, due to noise factors, phase variance exhibitsthe following behavior in absence of periodic scene patterns:Var(∠Y _(0, . . . ,M-1) [k])→0, k=k _(BL),Var(∠Y _(0, . . . ,M-1) [k])>>0, k≠k _(BL)  (1)where kϵ[_(low),k_(high)] and k_(low) and k_(high) denote the lowest andhighest possible vertical frequencies of the periodic horizontal bands,respectively. Consequently, when k_(BL) is unknown, it can be identifiedby determining which frequency component has minimal phase varianceacross all columns, i.e.

$\begin{matrix}{k_{BL} = {\underset{k \in {\lbrack{k_{low},k_{high}}\rbrack}}{argmin}\left( {{Var}\left( {\angle\;{Y_{0,\ldots\mspace{14mu},{M - 1}}\lbrack k\rbrack}} \right)} \right)}} & (2)\end{matrix}$Phase Parameterization

When measuring variations in phase, it is important to preserve phasecircularity. This is achieved by parameterizing real-valued phase values∠Y_(o, . . . , M-1)[k] using unit magnitude complex numbersp _(0, . . . ,M-1) ^(k)=exp(j·∠Y _(0, . . . ,M-1) [k]),  (3)where j is the imaginary unit. Subsequently, the variance inparameterized phase is given byVar(p _(0, . . . ,M-1) ^(k))=E[(p _(0, . . . ,M-1) ^(k) −E[p_(0, . . . ,M-1) ^(k)])·(p _(0, . . . ,M-1) ^(k) −E[p _(0, . . . ,M-1)^(k)])],  (4)where E[X] denotes the expected value of X, and X denotes the complexconjugate of X. The spatial frequency k_(BL) of the periodic horizontalbands is then given by

$\begin{matrix}{k_{BL} = {\underset{k \in {\lbrack{k_{low},k_{high}}\rbrack}}{argmin}\left( {{Var}\left( p_{0,\ldots\mspace{14mu},{M - 1}}^{k} \right)} \right)}} & (5)\end{matrix}$

FIG. 50 shows a plot of variance in parameterized phase vs. possiblemodulation frequencies, in the presence of a 755 Hz modulating source.Note that the dip 5002 at modulation frequency 755 Hz indicates thesignature of the DPR modulated signal.

Fourier Magnitude Peaks

Because periodic horizontal bands produced by the modulation signalextend throughout the whole frame, 2D Discrete Fourier Transform (DFT)of the bands yields two compact (narrow spectral support) peaks in DFTmagnitude along the vertical frequency axis. This fact is used inconjunction with phase consistency to increase detection accuracy andeliminate periodic scene patterns.

In order to filter out frequency components with compact magnitudepeaks, the magnitude spectrum in FIG. 51 is convolved with a 3×3 filterkernel with the general structure presented in FIG. 52.

The motivation for this choice of filter coefficients is to removemagnitude peaks of wide spectral support and replace them bynegative-valued dips. On the other hand, magnitude peaks of narrowspectral support remain positive-valued. FIG. 53 shows the result ofapplying the above filter kernel to vertical frequency components of themagnitude spectrum in FIG. 51.

Detection Algorithm

In what follows, f[m,n] denotes an M×N frame (M columns, N rows).{circumflex over (f)}[m,n] denotes a vertically-windowed version off[m,n].

1) Windowing

Obtain {circumflex over (f)}[m,n] by multiplying each columny_(0, . . . , M-1) of f[m,n] by an N-length Hann window.

2) ID DFTs

Compute N-length DFT for each column of the windowed frame {circumflexover (f)}[m,n] to obtain M 1D spectra Ŷ_(0, . . . , M-1).

Phase Parameterization

Parameterize the phase of Ŷ_(0, . . . , M-1) with accordance to Eq. 3 toobtain {circumflex over (p)}_(0, . . . , M-1)^(k)=exp(j·∠Ŷ_(0, . . . , M-1)[k]).

3) Phase Consistency

Compute the variance of {circumflex over (p)}_(0, . . . , M-1) ^(k)across all columns to obtain Var({circumflex over (p)}_(0, . . . , M-1)^(k)) with accordance to Eq. 4. This provides a phase consistencymeasure for every vertical spatial frequency k in the rangekϵ[k_(low),k_(high)].

4) Choose Candidate Banding Frequency Based on Phase Consistency

Determine the vertical spatial frequency with lowest varianceVar({circumflex over (p)}_(0, . . . , M-1) ^(k)) in the frequency range[k_(low),k_(high)]:

$\begin{matrix}{k_{cp} = {\underset{k \in {\lbrack{k_{low},k_{high}}\rbrack}}{argmin}\left( {{Var}\left( {\hat{p}}_{0,\ldots\mspace{14mu},{M - 1}}^{k} \right)} \right)}} & (6)\end{matrix}$

5) Dip Quality

Define left-handed and right-handed dip quality measures as

${q_{l} = \frac{{Var}\left( {\hat{p}}_{0,\ldots\mspace{14mu},{M - 1}}^{k_{cp} - 4} \right)}{{Var}\left( {\hat{p}}_{0,\ldots\mspace{14mu},{M - 1}}^{k_{cp}} \right)}}\mspace{14mu}$and${q_{r} = \frac{{Var}\left( {\hat{p}}_{0,\ldots\mspace{14mu},{M - 1}}^{k_{cp} + 4} \right)}{{Var}\left( {\hat{p}}_{0,\ldots\mspace{14mu},{M - 1}}^{k_{cp}} \right)}}\mspace{14mu},$respectively.

Define dip quality measure asq _(dip)=(q _(l) +q _(r)))/2  (7)

6) 2D DFT of Original Frame

Compute the M×N DFT of the original (non-windowed) frame f[m,n] toobtain F[k_(h),k_(v)]. Ensure that the DFT spectrum is centered aroundthe dc component.

7) Magnitude Filtering

Apply magnitude filtering, as described in Sec. 3, to vertical frequencycomponents. The result is a 1D filtered magnitude sequence|F′[0,k_(v)]|.

8) Choose Candidate Banding Frequency Based on Filtered Magnitude

Determine the vertical spatial frequency with highest filtered magnitudevalue in the frequency range k_(v)ϵ[k_(low),k_(high)]:

$\begin{matrix}{k_{cm} = {\underset{k_{v} \in {\lbrack{k_{low},k_{high}}\rbrack}}{argmax}\left( {{F^{\prime}\left\lbrack {0,k_{v}} \right\rbrack}} \right)}} & (8)\end{matrix}$

9) Rejection Criteria

If at least one of the following conditions is true, the frame underquestion is rejected and no “hit” is registered:

a. q_(dip) < 1.5 (low dip quality) b. |F′[0, k_(cm)]| < 0(negative-valued filtered magnitude) c. |k_(cp) − k_(cm)| > 2(significant disagreement between phase-based and magnitude-basedcandidate frequencies)Note that the above threshold values were experimentally determined toprovide good performance.

If none of the above conditions hold, define the low-precision estimateof the banding frequency as k_(BL, l)=k_(cm) and proceed to Steps 11-12or Steps 13-14.

10) Interpolated 1D DFTs (Zero-Padding)

Columns ŷ_(0, . . . , M-1) of the windowed frame {circumflex over(f)}[m,n] are zero-padded to length N′>N, yielding M N′-length columnsŷ_(0, . . . , M-1)′, Choice of N′ depends on the desired level ofprecision in banding frequency detection. Apply Step 2 toŷ_(0, . . . M-1)′ to compute M N′-length spectra Ŷ_(0, . . . , M-1)′.

11) Phase Consistency in the Neighborhood of Low-Precision Estimate

Let

$k_{a}^{\prime} = {{{{round}\left( {N^{\prime}\frac{k_{{BL},l} - 4}{N}} \right)}\mspace{14mu}{and}\mspace{14mu} k_{b}^{\prime}} = {{{round}\left( {N^{\prime}\frac{k_{{BL},l} + 4}{N}} \right)}.}}$Apply Steps 3-5 to Ŷ_(0, . . . , M-1)′[k′] specifically for thefrequency range k_(a)′≤k′≤k_(b)′.

Define the high-precision estimate of the banding frequency as

$\begin{matrix}{{k_{{BL},h} = {\underset{k^{\prime} \in {\lbrack{k_{a}^{\prime},k_{b}^{\prime}}\rbrack}}{argmin}\left( {{Var}\left( {\hat{p}}_{0,\ldots\mspace{14mu},{M - 1}}^{{\prime k}^{\prime}} \right)} \right)}},{{{where}\mspace{14mu}{\hat{p}}_{0,\ldots\mspace{14mu},{M - 1}}^{{\prime k}^{\prime}}} = {{\exp\left( {{j \cdot \angle}\;{{\hat{Y}}_{0,\ldots\mspace{14mu},{M - 1}}^{\prime}\left\lbrack k^{\prime} \right\rbrack}} \right)}.}}} & (9)\end{matrix}$

12) Interpolated 1D DFT (Zero-Padding) of Row-Averaged Intensity

Compute the average intensity of each row in the windowed frame{circumflex over (f)}[m,n] to obtain ŷ_(avg). Zero-pad ŷ_(avg) to lengthN′>N, yielding an N′-length column ŷ_(avg)′. Choice of N′ depends on thedesired level of precision in banding frequency detection. Compute theN′-length DFT of ŷ_(avg)′ to obtain Ŷ_(avg)′.

13) Peak Magnitude in the Neighborhood of Low-Precision Estimate

Let

$k_{a}^{\prime} = {{{{round}\left( {N^{\prime}\frac{k_{{BL},l} - 4}{N}} \right)}\mspace{14mu}{and}\mspace{14mu} k_{b}^{\prime}} = {{{round}\left( {N^{\prime}\frac{k_{{BL},l} + 4}{N}} \right)}.}}$Define the high-precision estimate of the banding frequency as

$\begin{matrix}{k_{{BL},h} = {\underset{k^{\prime} \in {\lbrack{k_{a}^{\prime},k_{b}^{\prime}}\rbrack}}{argmax}\left( {{{\hat{Y}}_{avg}^{\prime}\left\lbrack k^{\prime} \right\rbrack}} \right)}} & (10)\end{matrix}$

14) Convert Banding Frequency to DPR Modulation Frequency

Let ρ_(s) denote the row sampling frequency of the rolling shutter imagesensor. The DPR modulation frequency β corresponding to k_(BL, h) isthen given by

$\begin{matrix}{\beta = {\rho_{s}{\frac{k_{{BL},h}}{N^{\prime}}.}}} & (11)\end{matrix}$ρ_(s) can be determined experimentally by calibrating detection resultsobtained for a known modulation signal.Camera Switching

When utilizing mobile device receiver 103 to receive DPR modulatedsignals as part of a light positioning system, one aspect of the processis to identify which sensors to use. Because many mobile devices 103contain multiple sensors (for example, consider smartphones that containboth front and rear cameras), some embodiments include an algorithm todecide which camera sensor to use at a particular point in time. Forexample, consider the situation in FIG. 47, where a user 4702 a iswalking underneath light source 4401 a, emitting a DPR modulated spotsignal onto the floor 4701. The user's mobile device 4703 a containsboth a front and rear camera. As the user walks past the DPR illuminatedspot 4701, the rear camera is the first sensor to capture the image.When the user walks further along, the front facing camera is positionedunderneath the light. In this scenario, some embodiments use analgorithm to decide which camera to use when receiving the lightsignals.

FIG. 55 contains an implementation of a smart camera switchingalgorithm. Select sensors 5501 first decides which sensors to use. Thedecision for what sensor to use depends on a number of factors. In oneembodiment, the mobile device could use information about itsorientation, using sensor readings gathered from the gyroscope, compass,accelerometer, or computer vision, and then use that to determine whichcamera to choose. For example, if the device were to recognize that itwas oriented such that the rear camera was facing the ceiling, selectsensors 5501 would first choose the rear sensor to receive the beaconbased light positioning signal, as opposed to defaulting to the frontcamera. After selecting a sensor 5501, initialize sensors 3001 sets therequired hardware and software sensor parameters as described in FIG.30. Once the sensors have been selected and initialized, Tone (s)Detected? 5502 implements a DPR demodulation algorithm to analyzeincoming image/video frames for the presence of DPR tones. An output ofthe DPR demodulation algorithms is a confidence score representing thelikelihood that a DPR tone is present in the received images. If Tone(s)Detected? 5502 returns TRUE, then the algorithm terminates. If Tone (s)Detected? 5502 returns false, Continue Checking? 5503 decides whether ornot to try the remaining camera sensors. The decision is governed byfactors including battery life, location refresh time, and accuracyrequirements. If Continue Checking? 5503 continues the algorithm thecycle repeats, otherwise it is terminated.

Hiding Camera Preview

The primary functionality of most image sensors on mobile devices isrecording multimedia content in the form of video or photographs. Inthose use cases, it is desirable to display a preview feed of what thecamera is recording before actually shooting the video/photo. On mostmobile device's 103, displaying this preview feed is the default modewhen using the camera. On some mobile device API's, the softwareactually requires the application to display a preview feed such that ifa feed is not being displayed, the device is not allowed to record data.

For a mobile device 103 that is receiving DPR modulated light signals,it is desirable to hide the camera feed during DPR demodulation. Forexample, consider the case of a user walking around a retail store,using an in-store map to discover where items are within the store. Ifthe mobile device application framework required the camera preview tobe displayed, it would interrupt the user's interaction with the mobileapp. Furthermore, since there are a number of different camera tweaksthat can be applied during DPR demodulation (for example, modifying theexposure, focus, zoom, etc.), the camera preview image quality would below.

The present disclosure has explored a number of workarounds to therestrictions imposed by device APIs around the camera previewrequirements. In one embodiment, the mobile device creates a surfacethat is a single pixel in size, and then passes this surface as thecamera preview. The surface is an area on the mobile device fordisplaying information to a user. This fulfills the requirement of themobile device API that it is presented a surface to write to. Becausethe surface is only a pixel wide, the present system effectively tricksthe API, since the mobile device user cannot see the individual pixelssurface. In another embodiment, the mobile device simply does not createa camera preview surface.

Location Dependent Demodulation Algorithms

One aspect of DPR demodulation algorithms is that the performance ofsuch algorithms varies from location to location. For example, considera situation in which the floor of a building contains stripes or othernoise generating artifacts. The presence of artifacts on the floor addssignificant noise into the DPR signal. As discussed previously, both thefront and the rear camera can be used to recover DPR modulated signals.However, in a situation with the presence of noise on the rear camera,it would undesirable to use the rear camera at all. To account forsituations such as this, the present disclosure could uselocation-dependent demodulation algorithms. Using algorithm performanceinformation gathered from mobile device users 4402 a-b, the remoteserver 703 maintains records of which algorithms perform the best invarious situations. These algorithms also can be tied to specific deviceprofiles, such that different mobile phones utilize differentlocation-dependent algorithms. Furthermore, the location-dependentalgorithms also take into account which camera is being used. Typically,the performance characteristics for algorithms vary depending on theimage sensor type and the position on the mobile device. In the case oftypical smartphones, it is desirable to use a different algorithm on thefront versus the rear. For example, in one embodiment of the presentdisclosure the phase variance algorithm has better performance on thefront camera versus the rear camera.

A method for intelligently choosing the appropriate demodulation to usein a given scenario is presented in FIG. 54. First, an image sensor isselected and sampled 5401. This image sensor could be one of manysensors on the device (for example, either the front or rear sensor). Analgorithm for actually selecting the sensor is presented in FIG. 55.After the sensor is selected, a demodulation algorithm is chosen 5402.There are a variety of factors that can go into selecting theappropriate demodulation algorithm. The most important parameters arewhich image sensor is currently being used, and which location themobile device 103 is within. The mobile device 103 can send a request toa remote server 703 to find out which algorithm is appropriate for itscurrent environment. Upon selecting an algorithm, the algorithm isimplemented and the quality of the signal is reported 5403. The qualityof the signal is measured using a number of different metrics, asdescribed previously. After the quality of the signal has beendetermined, the algorithm might be adjusted 5404. Adjustments to thealgorithm could include changes to algorithms parameters such as thesampling rate, exposure time, or other such parameters as describedpreviously. The adjustments also can include changing the actual choiceof algorithm.

After the algorithm has been adjusted 5404, MovementDetected/Environment Change? 5405 is used to determine whether or notthe mobile device has changed its location. If the mobile device changeslocation, the image sensor is initialized and the sampling algorithmresamples at the beginning.

Interpolation

When using a light positioning system, it is often the case that amobile device receiver can receive signals from multiple lightssimultaneously. Consider the situation presented in FIG. 57. Mobiledevice user 5701 is within view of light spots 5702 a, 5702 b, and 5702c. Because the user 5701 is not positioned directly underneath anyparticular light, it is desirable to use the relative signal strengthsfor each light to interpolate the position of the mobile device receiver5703 from the lights. A mobile device 5703 detects quality signals fromtwo light sources 5702 a and 5702 b. For each quality signal, themagnitude of the signal is taken into account before resolving the exactlocation. A simple algorithm is to take the average of the lightlocations, each weighted by the total signal strength. For a given setof n magnitudes m_(a) and light coordinates x_(a), the position of themobile device 5703 is x, where:

$x = \frac{\sum_{a = 0}^{n}{x_{a}m_{a}}}{\sum m}$

Each coordinate is weighted against the total sum of signal strengths aspart of the averaging process. The result is a set of coordinates thatindicate the user's position.

In addition to weighting the total signal strength, the mobile device5703 also can use information gathered from multiple image sensors, aswell as information about the geometry of the room and the orientationof the device, to improve the positioning calculation. At this point,the device will have resolved a separate coordinate for the rear camera,and for the front camera. The device lies somewhere in between thesecoordinates. Using the geometry in FIG. 6 described U.S. patentapplication Ser. No. 13/526,773 filed Jun. 19, 2012 entitled “Method AndSystem For Digital Pulse Recognition Demodulation,” the entire contentsof which are hereby incorporated by reference, where x_(f) is thecoordinate resolved from the front camera, x_(r) is the coordinateresolved from the rear camera, and r_(h)=h₁/h₂ is the ratio of theheight of the device to the height of the ceiling, we can resolve theactual coordinate of the device, x_(a) using:x _(a) =r _(h)(x _(f) −x _(b))+x _(b)Repeat Identifiers

One potential issue that arises in a light based positioning system isthe possibility of repeat light identifiers. Because the identificationcode on each light source is not guaranteed to be globally unique,additional techniques can be to resolve which identifier is currentlybeing identified. In order to “narrow down” the possible candidates forwhich light bulb is being currently detected by the camera, an initial,broad location can be determined using geofencing. This is done via theGPS sensor of the device, or alternative sensors such as WiFi,Bluetooth, ultrasound, or other location technologies that would bereadily be understood by a person of ordinary skill in the art. Once thelocation sensor returns a location, a list of nearby light sources andtheir corresponding identification codes is retrieved. These lights willbecome the “candidates.” Any identification code detected by the mobiledevice's 103 image sensor will be compared against this list ofcandidate lights in order to determine the device's location. Thisprocess happens every time the mobile device 103 detects a majorlocation change.

FIG. 56 presents an algorithm for resolving between multiple lightidentifiers. Acquire Candidates 5601 fetches a list of nearby lightidentifiers. The list of nearby identifiers becomes the candidates newlight identifiers will be compared to. Examine recent identifiers 5602then looks at the list of light identifiers that the mobile device 103has recently seen. Determine distance from candidate 5603 then computesthe distance between the current recently seen identifier and the secondmost recently seen identifier. This distance is defined as thedifference between the signals. The concept is quite similar to aHamming Distance, which measures the minimum number of substitutionsrequired to change one code-word to another. Each of these candidatedistances are stored in a buffer 5604. Candidates Remaining? 5605continues to evaluate light signatures until all the candidates havebeen added to the candidate distance buffer. After all the signals havegathered and their distances are computed, Analyze Candidate Set 5606looks at the candidate signals, their distances, and then ascertainswhich candidate is most likely.

Quality Score Determination

A challenge when using a light based positioning system is determiningwhether or not a current signal contains the light identificationsignature. In order to quantify this, the present disclosure contains aquality score that measures the quality of the current detection. Thisquality score is analogous to a signal to noise ratio. A signal to noiseratio can be defined as the ratio of the power between a signal and thebackground noise. This can be measured in a number of ways, includingtaking the average power of each, the root mean square (RMS) of therespective powers, or the ratios in decibels. The present disclosurecontains multiple methods for determining signal quality. In oneembodiment, the quality score can be defined as the ratio of magnitudesof peaks of the signal spectrum. An algorithm first checks that allpeaks of the spectrum are above a specified threshold, and then measuresthe ratio of the highest peak to the second highest peak. Anothercomponent of the quality score can be the distance of the peaks from theexpected location of the peaks. For example, if a mobile device 103 weredemodulating a DPR signal that was known to use the frequencies 1000 Hz,1100 Hz, and 1200 Hz, and the receiver detected a signal of 800 Hz, themobile device receiver's quality score would take into account the 200Hz difference between the detected frequency 800 Hz and the closestknown frequency of 1000 Hz.

In addition to using analytical techniques such as ratio of peaks anddistance between peaks, quality score determination also can beperformed using pattern recognition algorithms. Pattern recognitionalgorithms are concerned with providing a label for a given set of data.Such algorithms output a confidence score associated with their choice,which can be used as the quality metric. If the qualitymetric/confidence score is too low, then the algorithm flags the resultas not containing a signal. In the context of a light positioningsystem, this refers to a situation where the output of the demodulationalgorithm is ignored. There are a number of techniques for patternrecognition, including neural networks, Bayes classifiers, kernelestimation, regression, principal component analysis, Kalman filters, orother such algorithms that would be readily understood by a person ofordinary skill in the art.

Lighting Control System

In a variety of use cases, it is desirable to control certain aspectssuch as the brightness, color, schedule, or direction of a light. Thereare a number of technologies for lighting control, including bothwireless systems such as Zigbee, WiFi, and 6LowPan, as well as wiredtechnologies such as DMX or other standard lighting control protocols.An important component of a lighting control system is the ability toknow the physical locations of the lights such that they can becontrolled.

For example, if a user wanted to remotely control a light in aparticular room, the central controller would need to know the locationsof all lights within a building. Furthermore, a user might want to onlycontrol lights in their direct vicinity. Consider the situation in FIG.58, where a mobile device user 5803 is standing underneath DPR enabledlights 5802 b and 5802 c. Each light 5802 a-c is broadcasting a DPRmodulated signal, which mobile device 5804 detects using its sensors.The mobile device then computes its indoor location, using thealgorithms discussed previously. The mobile device user 5803 is grantedpermission to control the lights 5802 a-c around them. In one embodimentof the present disclosure, the mobile device user has a “lightingprofile” that is associated with them. This lighting profile is used toadjust the lights in the room automatically, depending on the user'spreferences.

Lighting Control System Commissioning

A common problem when configuring a lighting system is identifying thephysical locations of lights for the purposes of lighting control. Thereare two steps when configuring a networked lighting control system. Onestep is network discovery, which is the problem of identifying deviceson the lighting network. In one embodiment, based on wired connections,devices on the network may be connected through power line communication(PLC), Ethernet, fibre optics, or other wired communication as would bereadily understood by one of ordinary skill in the art. In anotherembodiment, the light sources can be connected wirelessly usingprotocols such as WiFi, Zigbee, Bluetooth, 6LowPan, or other protocolsthat would be readily understood by a worker skilled in the art. In bothembodiments, the problem of network discovery is accounted for in theunderlying protocol, during which each device within the network isassigned a unique identifier.

The second step of configuring a networked lighting control system isdetermining the physical location of each light within the system. Thereare a number of existing techniques for doing this, all of which rely onlots of manual labor on the part of the individual tasked withperforming the commissioning. The present disclosure provides a methodby which beacon based light sources broadcast a self-identifying patternthat is recognizable by a mobile device receiver. This self-identifyingpattern can be used to assign the location of the light source whencommissioning a light positioning system. In one embodiment, the signalis demodulated using digital pulse recognition (DPR) techniques on animage sensor present on mobile device 5804.

Consider the situation presented in FIG. 58. Mobile device user 5803stands underneath light sources 5802 a-c. Each light source 5802 a-c isbroadcasting DPR modulated signals, and is connected to a network 5801.The lights be can networked using any standard networking technique. Asthe mobile device user 5803 receives information through the lightsignals, they can use the received signals to commission the system.Mobile device user 5803 receives the light signals, and then assignsthem a physical location within a building. The mobile device user cando the physical position assignment using a user interface like the onedescribed in FIG. 48, or via other inputs that would be readilyavailable.

For a networked lighting system, like the one presented in FIG. 58, itis desirable to provide a mobile device user 5803 with the ability tocontrol the lights both locally and remotely. If the lights arenetworked, the mobile device user 5803 could control the lights byselecting them manually via the user interface terminal. The lightscould also adjust depending on the time of day, occupancy sensors,current energy prices, ambient light readings, or user preferences.Transmitting an identifier through a modulated light 5802 c, which couldbe modulated using DPR modulation, to a mobile device 5804, allows themobile device user to directly control the light source. For example,consider the case where mobile device user 5803 is standing underneathlight source 5802 c, receiving a light identifier for the light source.The mobile device user could then send the identifier to a lightingcontrol module, along with a set of instructions to control the lightsource 5802 c. This set of instructions could include the color,brightness, light direction, time varying signal, on/off schedule or anyother parameters of the light source that a user would control. Thelight identifier could either be the network identifier of the lightsource (for example, the IP address if the light source were controlledover TCP/IP), or an identifier that could be correlated with the lightsource through the lighting controller.

Smart Light Bulb Reporting

Acquiring metrics for light sources is typically a very cumbersomeprocess. In most cases, a person wishing to measure light source metricsneeds a special handset equipped with a multitude of sensors. Thisreporting process is labor intensive, expensive, and limited in theamount of data that it can gather. Data about how a light sourceperforms over time is of large interest to light source manufacturers,who want to track how their products perform outside of their internaltest facilities.

There are a number of metrics that a bulb may want to report. Theseinclude total time on, age, color shift, temperature history, currentfluctuations, voltage fluctuations, individual LED performance,barometric pressure readings, dimming statistics, color statistics,manufacturer origin, parts included within the bulb, network status, orother such information that could be connected to the light source.

FIG. 46 presents a system by which a light source uses a light basedcommunication channel to report information about itself. A processingunit 4602 interfaces with sensors 4603. Possible sensors includetemperature, color temperature, electrical, barometric, network, or anyother sensor that could be added by one skilled in the art. Theprocessing 4602 acquires readings from the sensors at regular intervals,and then broadcasts those readings through modulator 4607. Modulator4607 is responsible for modulating the light output of light source 103.In one embodiment of the present disclosure, modulator 4607 is a DPRmodulator, as described previously in FIG. 21. Data 4602 is passed fromsensors 4603 to processing unit 4602, before being sent to Encoder 2102.The packet structure could be determined either in the processing unit4602 or the modulator 4607. Within the packet, in addition to thestandard start bits, data bits, and error bits, there could also bemetadata tags that indicate the type of information in the packet. Forexample, there could be a special header that would indicate that thepacket contains information about bulb longevity. The header couldcontain the length of the bulb longevity portion, or the length could beset as a predetermined portion. The header could also include anidentifier to indicate which packet was being transmitted in a series ofpackets. This can be used for situations in which multiple pieces ofdata are being transmitted in succession. The design of a packet is wellunderstood by workers of ordinary skill in the art. The presentdisclosure includes a packet structure that contains an identifier forthe light source, light source dimming information, longevity,temperature, and color temperature.

A timestamp is added to the data by the mobile device receiver 103.After the mobile device receiver completes demodulation of the packet,the receiver can either send the data to a remote server for storage, orstore it locally. In the case of a poor network connection, the receiverdevice 103 would store the light source information before transmittingthe data to a server once a sufficient connection is formed. In thisway, mobile device receivers can crowd source the collection ofinformation about light source performance, without having to do manualsurveying. This could be done in conjunction with light sourcesproviding an indoor positioning service.

Bar Code Scanner

A common problem for employees of retail stores, hospitals, warehouses,or other such locations is physically tagging the location of objects,products, and supplies. For example, retail employees are frequentlytasked with scanning barcodes when performing inventory in store.Typically, after an employee scans a code, they must manually input thelocation of the product into the system. To speed up this process, thepresent system utilizes light based positioning in conjunction withbarcode scanning. FIG. 59 contains a depiction of a mobile device user5901 standing underneath light sources 4401 a-c, each broadcasting amodulated light signal. When the mobile device user successfully scans aproduct code 5902, the product code is uploaded to a remote server alongwith its current position, which is derived using light positioning.This reduces the manual labor required on the part of the mobile deviceuser. The product code could be a bar code, QR code, product photo,RF-ID, photo, or any other tag that would be readily understood by aperson skilled in the art.

Alternative Sensors for DPR Demodulation

Using an image sensor which utilizes a rolling shutter mechanism forexposing the image is a convenient method for performing DPRdemodulation on today's mobile devices. This is because the vastmajority of devices contain CMOS sensors for capturing multimediacontent. However, other image sensors can be used. The CMOS sensor isconvenient because the rolling shutter functionality acts as atime-domain sample of the illuminated DPR signal. However, thisfunctionality can be replicated on any type of optical sensor. Forexample, if instead of rolling shutter CMOS a global shutter chargecoupled device “CCD” sensor was used, the sampling function would simplychange from a single frame to multiple frame analysis. Light intensityfluctuations of individual pixels on the recovered image can be sampledacross multiple frames. If a photodiode were used as the sensor, thephotodiode would be sampled by an analog to digital converter and thensent to a processing unit. In all of these cases, the only aspect thatchanges is the response function of the receiving device. One ofordinary skill in the art would readily understand the requirements tosupport DPR demodulation on alternative receivers, including but notlimited to photodiodes and global shutter image sensors.

DPR Enclosure Module

FIG. 60 describes a physical DPR enclosure 6001 which contains the DPRmodulator 6004 and stress relieved 6003 a-b incoming connections 6002and outgoing connections 6005.

The techniques and methods disclosed for use in light based positioningsystems can be used with a variety of camera equipped mobile orstationary devices, such as: mobile phones, tablet computers, netbooks,laptops, desktops, or custom designed hardware. Further, the scope ofthe present invention is not limited to the above described embodiments,but rather is defined by the appended claims. These claims representmodifications and improvements to what has been described.

Electrical Design

FIG. 61 depicts a Self-Identifying Optical Transmitter 6101 inaccordance with some embodiments, which transmits an identifier via anoptical signal. In some embodiments, the Self-Identifying OpticalTransmitter 6101 may also transmit additional data using the opticalsignal. Processing Unit 6104 sends a signal to modulator 6105. ProcessorUnit 6104 can come in the form of a microcontroller, FPGA, series oflogic gates, or other such computing element that would be readilyunderstood by a person of ordinary skill in the art. The processing unitcan also be a series of logic elements/switches controlled via analogcircuitry. To minimize power consumption, the processing unit can beconfigured to operate in a low power state. Minimizing power consumptioncan be important in the case of a battery-powered system. TheSelf-Identifying Optical Transmitter 6101 receives power from PowerSource 6103. In some embodiments, Power Source 6103 is a battery-poweredsystem. Power Source 6103 can also be in the form of an external powersource. The Processing Unit 6104 contains a Data Interface 6109, whichcan be used to program the device, exchange data, or load information tobe transmitted across Light Output 6110. An executable program forProcessing Unit 6104 may be pre-loaded before the Self-IdentifyingOptical Transmitter 6101 is deployed. This program may be in the form ofone or more software modules. The processing unit contains anon-volatile memory storage area 6102 for storing information to betransmitted across the Light Output 6110. This information can includean identifier for the Light Output 6110. The identifier can be aquasi-unique identifier, or globally unique if the identifiers werecentrally-managed or pre-configured. This quasi-unique random identifieris received on the mobile device, which then verifies that the mobiledevice came into close proximity with the self-identifying one-wayoptical transmitter. In some embodiments, the data transmission mayinclude data in addition to the identifier, such as meta-data. Forexample, the optical data transmission stream can include informationabout the current battery state of the device. This information can beused to inform the owner of the device that it is time to change thebattery.

Modulator 6105 is used to control the power transmitted through thelight source, thus driving the optically encoded signal in Light Output6110. Modulator 6105 can come in the form of a MOSFET, BJT, transistorswitch, discharge circuit, or any such device for controllingcurrent/voltage that would be understood by a person of ordinary skillin the art.

Light source 6106 may take an electrical input from modulator 6105 andconvert it into an optical signal. In some embodiments, light source6106 may include light emitting diodes (LEDs). LEDs may becurrent-driven devices, in which case the brightness of the LED isproportional to the current being driven through it. In otherembodiments, the LEDs may be voltage-controlled, or other light sourcesaside from LEDs may be used that have different current/voltagecharacteristics. The light sources pulse the electrical signal receivedfrom the modulator in an optical format.

Optics 6108 disperse the optical signal 6107 emitted from the lightsource 6106 to improve the ability of a mobile device 6203, asrepresented in FIG. 62, to receive and interpret the Light Output 6110.There are a number of reasons for this. For example, in the case oflight source 6106 including LEDs, the optical signal 6107 may be emittedfrom a single point source. Because it is desirable to allow a mobiledevice user to quickly scan their mobile device past theself-identifying optical transmitter, increasing the surface area of theoptical signal may helpfully reduce the time used by a mobile deviceuser to align their mobile device with the light source 6106.Furthermore, because mobile devices 6203 can come in many different formfactors, it is desirable to have a wide surface area in order to supportmany different mobile devices. Dispersing the optical signal alsoreduces the likelihood that the brightness of individual point LEDsources will wash out the image sensor on mobile device receiver 6203.Furthermore, viewing an LED with the naked eye without dispersing thelight may annoy users.

FIG. 62 represents the use of self-identifying optical transmitters bymuseum patrons in a museum space, in accordance with some embodiments.This representation includes a museum space 6205, museum patrons 6201and 6204, mobile device 6203 held by museum patron 6204, andself-identifying one-way optical transmitters 6202 a-6202 d. In theseembodiments, museum patrons 6201 and 6204 equipped with mobile devicesbrowse a museum space 6205 and may desire additional information aboutthe exhibits of the space. Self-identifying one-way optical transmitters6202 a-6202 d may be positioned throughout the exhibit 6205. Thesetransmitters may be strategically placed near points of particularinterest, near points frequently visited by museum patrons, near pointsthat are easily accessible by users of mobile devices, or in any otherconfiguration within a space. The self-identifying optical transmittersmay be mounted or placed vertically, horizontally, or at any other anglewithin the space. The self-identifying optical transmitters may bemounted or placed on walls, pedestals, displays, ceilings, or any otherplace visible by users of mobile devices. Self-identifying opticaltransmitters may also be positioned in other types of spaces, such asgrocery stores, shopping malls, or transit stations. One or manyself-identifying optical transmitters may be placed in these sites.

Within the museum space 6205 that is equipped with self-identifyingoptical transmitters 6202 a-6202 d, museum patron and mobile device user6204 may tap their mobile device 6203 onto the self-identifying one-wayoptical transmitter 6202 d. The museum patron may do this because he orshe desires more information about the portion of the museum space inwhich he or she is located, because he or she wishes to report an errorabout the space, or for any other reason. This self-identifying one-wayoptical transmitter 6202 d is mounted on a wall within the museum space6205, and the light output from the transmitter may be received by animage sensor of the mobile device 6203. A user does not necessarily needto tap their mobile device onto one of the self-identifying opticaltransmitters, but instead may place their mobile device in any positionwhere it is able to receive the light output from a self-identifyingoptical transmitter. Upon receipt of the light output, the mobile device6203 may interpret the light output as a code. This code can be used topull up additional content associated with the current location ofself-identifying optical transmitter 6202 d within the museum space6205. In the case of a retail store, the code may be used to depict anadvertisement, acquire a coupon, receive credit for visiting a location,order an item, call for customer service, or pull up an online orderform. Any other information by be received and interpreted by the mobiledevice 6203 and then related to the location of the mobile device 6203.

In addition to using a self-identifying transmitter such as the onedescribed in FIG. 61 to receive information, as described above, amobile device user 6201 or 6204 may use the self-identifying opticaltransmitters 6202 a-6202 d to assign content to a particular location.For example, a common scenario when taking photographs is to tag thephotograph to a particular physical location. The EXIF format, whichdefines meta-data tags that are added to images, specifically builds ina standard set of tags for location information. These tags are commonlyadded when taking photographs with a GPS enabled mobile device.

A mobile device user can utilize, for example, the self-identifyingtransmitter 6202 a to geo-tag media, in the same way that GPS is used togeo-tag a picture. Note that the word geo-tag refers to associating apiece of digital media with a particular location. This location can bederived by the self-identifying transmitter, or by using positionderived from light as described above. In the case of mobile device user6201 taking a picture, this user may first tap their mobile device ontoself-identifying transmitter 6202 a and thus associate that picture withthe transmitter 6202 a. The picture can then be uploaded to a remoteserver, with the corresponding identifier used to associate the picturewith the self-identifying transmitter 6202 a. In some embodiments, thegeo-tagging information may be derived from an overhead lightpositioning system, such as the one described previously in U.S. patentapplication Ser. No. 13/526,773 filed Jun. 19, 2012 entitled “Method andSystem For Digital Pulse Recognition Demodulation.” Note that the mediais not necessarily limited to photos, but can take the form of audiorecordings, movies, video, user generated text, or any such media thatcan be generated.

FIG. 63 is a representation of a self-identifying optical transmitter6301. The transmitter includes light sources 6303 a and 6303 b that emitlight towards a surface 6302. The self-identifying optical transmitter6301 may be placed in a museum, a retail store, or any other locationwhere the optical signal emitted by the light sources may be accessibleby a mobile device user. Although FIG. 63 depicts one embodiment of aself-identifying optical transmitter, a variety of housing shapes,material types, and light source placements may be used for altering orimproving the optical performance of the self-identifying opticaltransmitter 6301. A different number of light sources may also be used.In some embodiments, the light sources may be LED point sources. Inthese and other embodiments, the light sources may be are reflected offof a surface 6302. Reflecting the light from the light sources in thisway may increase the area across which the light may be received. Thesurface is ideally colored white, but can be any color that sufficientlyreflects the light from light sources 6303 a and 6303 b. The lightsources 6303 a and 6303 b can also have lensing on them in order todisperse the optical signal they emit. The lensing may be placeddirectly over the light sources, or on some other portion of theself-identifying optical transmitter. For example, in the case of a redlight source being used, the surface can be colored in a different wayin order to increase the reflectiveness. Note that the surface 6202 canbe angled, flat, rounded, or another such shape that would be understoodby a person of ordinary skill in the art. In some embodiments, a lensmay be placed at an angle to surface 6202 such that the light sourcesare enclosed within the self-identifying optical transmitter and thelens.

FIG. 64 is another representation of a self-identifying opticaltransmitter 6401, according to some embodiments. The self-identifyingoptical transmitter 6401 may contain one or more light sources (notshown) and be enclosed by lensing 6402 to disperse the light from thelight sources. A mobile device user may place their mobile device nearthe self-identifying optical transmitter 6401 or tap their mobile deviceonto the lensing 6402 itself. The shape of the lens may be round, flat,or any other shape that disperses the light source. Frosted lenses are apopular and effective choice for light dispersion, but any lens may beused. In some embodiments, lensing can be combined with a reflectivesurface within the self-identifying optical transmitter 6401 to improvethe signal dispersion characteristics even further. In yet otherembodiments, the top of the lens 6402 may be covered with a film orpartially transparent surface to further improve optical dispersion.

FIG. 65 depicts a self-identifying optical transmitter 6501 with arounded lens 6502, in accordance with some embodiments. As with theoptical transmitters described above, the self-identifying opticaltransmitter 6501 may contain on or more light sources and a user mayplace their mobile device or near the transmitter to receive an opticalsignal broadcast from the light sources. The rounded lens can be of anydimension. A rounded lens holds advantages in terms of appearance anduser experience. Since the rounded lens is raised, it forms an easiersurface onto which a user may tap their mobile device. This is becausethe flat form factor of a typical mobile device will rest tangential tothe curved surface of the rounded lens, making it unlikely that thecamera lens will rest directly on the surface of the lens. Someseparation between the lens and the surface is desirable to avoidwashing out the image sensor.

FIG. 66 illustrates another possible implementation of aself-identifying optical transmitter 6601, in accordance with someembodiments. The self-identifying optical transmitter 6601 includes fourlight sources 6602 a-6602 d arranged inside of a housing. A differentnumber of light sources may also be used. The light sources 6602 a-6602d may be LED light sources. These light sources project a signal withinand around the transmitter device. Having multiple light sources mayimprove the optical power of the transmitter. Arranging the light sourceat different position and angles within the self-identifying opticaltransmitter may increases the range and areas within which the opticalsignals broadcast may be received by a mobile device. However, someembodiments with multiple light sources have the disadvantage ofpossibly lowering the battery life because more light sources may draw ahigher current. Each light source 6602 a-d may contain individuallenses.

Conserving power, and reducing the light output for aesthetic purposes,is a desired feature of a self-identifying broadcast light source. Oneway to achieve this is by having the Processing Unit 6104 operate in alow power state. Another method for improving the lifetime and powerconsumption of a stand-alone optical-transmitter is to power cycle thedevice when it is not being used. There are a number of ways to achievethis. In some embodiments, there may be a proximity sensor on thetransmitter device 6101. This proximity sensor is responsible forpowering up the transmitter device 6101 when a user comes close to it.This ensures that the transmitter broadcasts the optical signal whenthere is a user nearby to receive it. Other embodiments have amechanical button or switch that the user taps to turn on the light.Other embodiments may include an inductive or capacitive circuit that istriggered upon contact with the mobile device 6101. All of thesepossible external inputs can connect to the self-identifying opticaltransmitter 6101 through data interface 6109.

Modulation

In some embodiments, digital pulse recognition (DPR) modulation is usedto modulate the light source 6106 to transmit the light output 6110. Asdescribed previously in U.S. patent application Ser. No. 13/526,773filed Jun. 19, 2012 entitled “Method and System For Digital PulseRecognition Demodulation,” DPR modulation pulses the light source with alight pattern. A mobile device receiver demodulates the signal byexploiting the rolling shutter mechanism on a CMOS image sensor. Thereceiver examines the incoming images for patterns encoded within thelight pulses.

In other embodiments, changing color patterns can be used to transmit anidentifier. In embodiments where light source 6106 includes RGB LEDs,the information may be encoded into a series of changing color patterns.A mobile device receiver 6203 can examine the pattern of color shiftsand decode the corresponding identifier. Note in both embodiments thatuse DPR modulation and embodiments that use color modulation, theinformation transfer is not necessarily limited to the transmission ofan identifier. Additional information can also be encoded into theseries of light pulses. This can include the battery state of thetransmitter, total time on, or data sent via Data Interface 6109. Notethat Data Interface 6109 can connect with any arbitrary data source.

In some embodiments, pulse width modulation (PWM) may be used totransmit an arbitrary signal. PWM is a desirable choice of modulatingLED sources because it reduces color shift in the LEDs. Because thecolor-shift of an LED changes depending on the amount of current that isdriven through it, it is desirable to transmit a fixed current throughthe LEDs, and control the brightness by varying the time the LED is“on.” For this reason, many LED light sources use PWM dimmingtechniques, as opposed to constant current techniques that can causecolor shift. By using PWM to construct an arbitrary waveform,information transfer can be facilitated across an optical channel whilepreserving the color quality of the LED output. PWM modulation is usedto transmit an arbitrary signal by varying the duty cycle of a pulse ata set frequency. The width of the pulse represents a quantization of theamplitude of the arbitrary signal at a point in time.

To facilitate the transfer of an arbitrary signal using PWM modulation,first a carrier frequency is chosen. The carrier frequency is thefrequency of the PWM wave. The choice of carrier frequency depends on anumber of factors. These include the quantization resolution of thetransmitter, the clock speed of the transmitter, and the sampling rateof the receiver.

FIG. 67 illustrates an example waveform for using PWM to transmit anarbitrary signal. As discussed above, this can be used to drive an LEDlight source. A carrier wave 6703, representing a PWM wave with a setfrequency, is used to represent an arbitrary signal 6702. At any pointon the arbitrary signal, the width of the pulse 6701 is used torepresent the amplitude of the signal 6702. In this manner, the PWM wavemay act as an analog-to-digital converter for an arbitrary signal. Forexample, consider the waveform presented in FIG. 67. The PWM carrierwave has a frequency of 64 Khz. The period of the wave is 1/64 Khz, or15.6 microseconds. At each point along the arbitrary signal 6702, theamplitude of the signal is converted to a corresponding pulse width. Soif the amplitude of the signal is 50% of its max value, the pulse widthmay be 50%. With a 64 Khz wave, this corresponds to a pulse width of 7.8microseconds. If the amplitude is 25% of its max value, the pulse widthmay be 25%, or 3.9 microseconds. In these embodiment, an analog signalis converted to a digital representation as a pulse width. Provided thatthe carrier frequency is chosen to be lower than the Nyquist rate of themobile device receiver, any arbitrary signal can be transmitted to themobile device.

In some embodiments, this PWM wave can be used to construct a sum ofsinusoids. The combination of frequencies can then be mapped to anencoding scheme to uniquely identify a light source, or transmit othersuch data. Possible modulation techniques that can be constructed fromPWM waves include Orthogonal Frequency-Division Multiplexing, Quadratureamplitude modulation, amplitude modulation, phase-shift keying, on-offkeying, pulse-position modulation, spread spectrum, or other suchmodulation techniques that would be understood by a person of ordinaryskill in the art.

On the receiver side, there are a number of techniques for improving thecapabilities of the mobile device. As described previously in U.S.patent application Ser. No. 13/526,773 filed Jun. 19, 2012 entitled“Method and System For Digital Pulse Recognition Demodulation,” imagesensor settings such as exposure, white balance, frame rate, andresolution may be modified in ways that improve the signal-to-noisecharacteristics of the receiver. In general, increasing the frame rateof the receiver increases the sampling rate, and consequently the datarate of the receiver. Furthermore, increasing the resolution of thereceived image also improves the receiver capabilities. This is due tothe discrete nature of the sampling process. In some embodiments, whenusing a Fourier transform based approach to recovering the transmittedsignal, increasing the resolution of the image sensor on a receivereffectively increases the number of samples. Increasing the number ofsamples when performing Fourier analysis may increase the frequencyresolution of the Fourier transform. In applications that utilize thetransmission of individual tones, or sets of tones, to identify lightsources, increasing the frequency resolution is important in increasingthe number of identifiers that can be transmitted. Another benefit ofincreasing the frequency resolution is that, in some cases, thisoverrides optimizations done at the image sensor level. For example, anumber of image sensors may compress an image before returning it to thehigher layers of the software stack. Because this compression oftenhappens at the driver level, some camera APIs may not be able tooverride the compression. Data compression can cause information withinthe image to be lost. In the case of a DPR modulated signal, this maycause the image to lose features that carry information transmitted viathe optical signal. Thus, increasing the resolution of the image sensor501 has the dual benefit of increasing the bandwidth of the receiver103, as well as reducing the possibility of information loss throughcompression and other optimizations at the image sensor level.

Applications

Associating the media with the self-identifying transmitter, or aposition derived from light, allows for users to overlay digital contentonto a physical space. Future visitors to the same location can scrollthrough the experiences of previous visitors to the space. Userswatching remotely would be able to view a live feed of activity in thespace, and see digital content as it is generated. As an example,consider the situation presented in FIG. 68. A high-brightness lightsource 6801 is broadcasting a modulated signal overhead. The signal 6802may be reflected off the clouds, or other large objects. Mobile deviceusers, such as user 6804, can point their devices 6803 at the reflectedsignal 6802, which is broadcasting an optical signal. Mobile deviceusers can then upload text, status updates, audio, video, pictures, orother such media, which is tagged with the identifier received via thevisible light signal. Remote users can then subscribe to a media feedthat is tagged with the particular physical location. Users who view thecloud reflected signal 6802 can also be redirected to a web link, whichcan present them with advertisements, directions to a nearby restaurant,deals, additional information about a venue, a ticketing system, orother such data that would be understood by a person of ordinary skillin the art.

A common usage scenario for one-way authentication systems is verifyingthat a user has visited a particular location. For example, consider thesituation in FIG. 69. Mobile device user 6902 is passing through acheckout aisle 6901. The user may be participating in a customer loyaltyprogram, where they receive a credit for walking through the checkoutaisle. In order to verify the user has actually gone through thecheckout aisle, a mechanism may be used to tie the user's location tothat particular location. The user 6902 may tap their mobile device 6904onto a self-identifying optical transmitter 6903 placed near the pointof interest. This transmitter 6903 transfers an identifier to the mobile6904, which stores the identifier to verify their position. Thisidentifier can be used on an application running locally on the device,or transmitted to a remote server. The identifier provides a securemethod by which the user's position may be verified. Combined withinformation about the user's identity, previous purchase history,demographics, and previous locations visited, the authenticated locationtransmission can be used to administer the customer loyalty program.Note that this approach has significant security advantages over QRcodes, because it is very difficult to replicate the optical signaltransmitted by transmitter 6903.

What is claimed is:
 1. A mobile device, comprising: an image sensor; awireless interface configured to communicate through a network over awireless medium; a processor coupled to the image sensor and thewireless interface; a memory; and software in the memory to be run bythe processor, wherein running of the software by the processorconfigures the mobile device to implement functions, including functionsto: operate the image sensor to capture one or more images including amodulated visible light signal transmitted from a visible light sourcelocated within a space, wherein the captured one or more images includea pattern of stripes; determine from the captured one or more images awidth of one or more stripes of the pattern of stripes; determine, basedat least in part on the determined widths of stripes in the obtainedpattern of stripes, information corresponding to a geographical locationof the visible light source; store the determined information; andprocess the determined information to verify that a user of the mobiledevice came into proximity of the visible light source.
 2. The mobiledevice of claim 1, wherein the function to determine informationcorresponding to the geographical location of the visible light sourcecomprises a function to obtain, based at least in part on the pattern ofstripes, an identifier of the visible light source.
 3. The mobile deviceof claim 2, wherein the function to demodulate the modulated visiblelight signal comprises a function to calculate a frequency content ofthe image based on a detected stripe width of respective stripes in thepattern of stripes.
 4. The mobile device of claim 2, wherein the imagesensor is a rolling shutter camera.
 5. The mobile device of claim 2,wherein the process function comprises a function to transmit, via thewireless interface, the identifier of the visible light source to aserver for user location verification.
 6. The mobile device of claim 5,wherein the function to transmit the identifier of the visible lightsource includes a function to transmit an identification of the user ofthe mobile device in conjunction with the identifier of the visiblelight source.
 7. The mobile device of claim 2, wherein the processfunction further comprises a function to obtain, via the wirelessinterface, an indication of the geographical location of the mobiledevice based on the identifier of the visible light source.
 8. Themobile device of claim 2, wherein the process function comprises afunction to retrieve, based at least in part on the identifier of thevisible light source and from the memory, an indication of ageographical location of the mobile device.
 9. The mobile device ofclaim 1, wherein the implemented functions further include a function tostore, in the memory, the determined information in conjunction with anindication of a geographical location of the user.
 10. A method,comprising steps of: capturing, via an image sensor of a mobile device,one or more images including a modulated visible light signaltransmitted from a visible light source located within a space, whereinthe captured one or more images include a pattern of stripes; obtainingthe pattern of stripes from the captured one or more images;determining, based at least in part on the obtained pattern of stripes,information corresponding to a geographical location of the visiblelight source; storing the determined information; and processing thedetermined information to verify that a user of the mobile device cameinto proximity of the visible light source.
 11. The method of claim 10,wherein the step of determining information corresponding to thegeographical location of the visible light source comprises a step ofobtaining, based at least in part on the obtained pattern of stripes, anidentifier of the visible light source.
 12. The method of claim 11,wherein the processing step comprises a step of transmitting, via awireless interface of the mobile device, the obtained identifier of thevisible light source to a server for user location verification.
 13. Themethod of claim 12, wherein the step of transmitting the identifier ofthe visible light source includes a step of transmitting anidentification of the user of the mobile device in conjunction with theidentifier of the visible light source.
 14. The method of claim 11,wherein the processing step further comprises a step of receiving, via awireless interface, an indication of a geographical location of themobile device based on the identifier of the visible light source. 15.The method of claim 11, wherein the processing step comprises a step ofretrieving, based at least in part on the obtained identifier of thevisible light source and from a memory, an indication of a geographicallocation of the mobile device.
 16. The method of claim 10, furthercomprising a step of storing, in a memory, the determined information inconjunction with an indication of a geographical location of the user.17. The method of claim 10, further comprising a step of transmitting,via a wireless interface, the determined information.
 18. The method ofclaim 17, wherein the step of transmitting the determined informationincludes a step of transmitting an identification of the user of themobile device in conjunction with the determined information.
 19. Anon-transitory tangible computer readable medium embodying instructions,wherein execution of the instructions by a processor of a mobile deviceconfigures the mobile device to implement functions, including functionsto: operate an image sensor to capture one or more images including amodulated visible light signal transmitted from a visible light sourcelocated within a space, wherein the captured one or more images includea pattern of stripes; obtain the pattern of stripes from the capturedone or more images; determine, based at least in part on the obtainedpattern of stripes, information corresponding to a geographical locationof the visible light source; store the determined information; andprocess the determined information to verify that a user of the mobiledevice came into proximity of the visible light source.
 20. The computerreadable medium of claim 19, wherein the implemented functions furtherinclude a function to transmit, via a wireless interface of the mobiledevice, the determined information in conjunction with an identificationof the user of the mobile device to a server for user locationverification.